Mean-Payoff Pushdown Games* 



Krishnendu Chatterjee 1 and Yaron Velner 2 

1 1ST Austria 
2 Tel Aviv University, Israel 



Abstract. Two-player games on graphs are central in many problems in formal verification and 
program analysis such as synthesis and verification of open systems. In this work we consider 
solving recursive game graphs (or pushdown game graphs) that can model the control flow 
of sequential programs with recursion. While pushdown games have been studied before with 
qualitative objectives, such as reachability and w-regular objectives, in this work we study for 
the first time such games with the most well-studied quantitative objective, namely, mean-payoff 
objectives. In pushdown games two types of strategies are relevant: (1) global strategies, that 
depend on the entire global history; and (2) modular strategies, that have only local memory 
" and thus do not depend on the context of invocation, but only on the history of the current 

invocation of the module. Our main results are as follows: (1) One-player pushdown games 
O | with mean-payoff objectives under global strategies are decidable in polynomial time. (2) Two- 

1 player pushdown games with mean-payoff objectives under global strategies are undecidable. 

(3) One-player pushdown games with mean-payoff objectives under modular strategies are NP- 
hard. (4) Two-player pushdown games with mean-payoff objectives under modular strategies 
can be solved in NP (i.e., both one-player and two-player pushdown games with mean-payoff 
objectives under modular strategies are NP-complete) . We also establish the optimal strategy 
complexity showing that global strategies for mean-payoff objectives require infinite memory 
even in one-player pushdown games; and memory less modular strategies are sufficient in two- 
player pushdown games. Finally we also show that all the problems have the same complexity 
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if the stack boundedness condition is added, where along with the mean-payoff objective the 
player must also ensure that the stack height is bounded. 
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Games on graphs. Two-player games played on finite-state graphs provide the mathematical frame- 
work to analyze several important problems in computer science as well as mathematics. In particular, 
when the vertices of the graph represent the states of a reactive system and the edges represent the 
transitions, then the synthesis problem (Church's problem) asks for the construction of a winning 
strategy in a game played on the graph [11, 30, 29, 28]. Game-theoretic formulations have also proved 
useful for the verification [3], refinement [23], and compatibility checking [14] of reactive systems. 
Games played on graphs are dynamic games that proceed for an infinite number of rounds. The ver- 
tex set of the graph is partitioned into player-1 vertices and player-2 vertices. The game starts at an 
initial vertex, and if the current vertex is a player-1 vertex, then player 1 chooses an out- going edge, 
and if the current vertex is a player-2 vertex, then player 2 does likewise. This process is repeated 
forever, and gives rise to an outcome of the game, called a play, that consists of the infinite sequence 
of states that are visited. Two-player games on finite-state graphs with qualitative objectives such 
as reachability, liveness, w-regular conditions formalized as the canonical parity objectives, strong 
fairness objectives, etc. have been extensively studied in the literature [22, 17, 18,35,31,21]. 

The extensions. The study of two-player finite-state games with qualitative objectives has been 
extended in two orthogonal directions in literature: (1) two-player infinite-state games with qualitative 
objectives; and (2) two-player finite-state games with quantitative objectives. One of the most well- 
studied model of infinite-state games with qualitative objectives are pushdown games (or games on 
recursive state machines) that can model reactive systems with recursion (or model the control flow 
of sequential programs with recursion). Pushdown games with reachability and parity objectives have 
been studied in [33, 32, 5, 4] (also see [19, 20, 10, 9] for sample research in stochastic pushdown games). 
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The most well-studied quantitative objective is the mean-payoff objective, where a reward is associated 
with every transition and the goal of one of the players is to maximize the long-run average of the 
rewards (and the goal of the opponent is to minimize). Two-player finite-state games with mean- 
payoff objectives have been studied in [16,36,27], and more recently applied in synthesis of reactive 
systems with quality guarantee [6] and robustness [7]. Moreover recently many quantitative logics 
and automata theoretic formalisms have been proposed with mean-payoff objectives in their heart to 
express properties such as reliability requirements, and resource bounds of reactive systems [12, 8, 15]. 
Thus pushdown games with mean-payoff objectives would be a central theoretical question for model 
checking of quantitative logics (specifying reliability and resource bounds) on reactive systems with 
recursion feature. 

Pushdown mean-payoff games. In this work we study for the first time pushdown games with 
mean-payoff objectives (to the best of our knowledge mean-payoff objectives have not been studied in 
the context of pushdown games) . In pushdown games two types of strategies are relevant and studied 
in literature. The first is the global strategies, where a global strategy can choose the successor state 
depending on the entire global history of the play (where history is the finite sequence of configurations 
of the current prefix of a play). The second is the modular strategies, and modular strategies are 
understood more intuitively in the model of games on recursive state machines. A recursive state 
machine (RSM) consists of a set of component machines (or modules). Each module has a set of 
nodes (atomic states) and boxes (each of which is mapped to a module), a well-defined interface 
consisting of entry and exit nodes, and edges connecting nodes/boxes. An edge entering a box models 
the invocation of the module associated with the box and an edge leaving the box represents return 
from the module. In the game version the nodes are partitioned into player-1 nodes and player-2 nodes. 
Due to recursion the underlying global state-space is infinite and isomorphic to pushdown games. The 
polynomial-time equivalence of pushdown games and recursive games has been established in [5]. A 
modular strategy is a strategy that has only local memory, and thus, the strategy does not depend 
on the context of invocation of the module, but only on the history within the current invocation of 
the module. In other words, modular strategies are appealing because they are stackless strategies, 
decomposable into one for each module. In this work we will study pushdown games with mean-payoff 
objectives for both global and modular strategies. 

Previous results. Pushdown games with qualitative objectives were studied in [33, 32]. It was shown 
in [33] that solving pushdown games (i.e., determining the winner in pushdown games) with reachabil- 
ity objectives under global strategies is EXPTIME-hard, and pushdown games with parity objectives 
under global strategies can be solved in EXPTIME. Thus it follows that pushdown games with reach- 
ability and parity objectives under global strategies are EXPTIME-complete. The notion of modular 
strategies in games on recursive state machines was introduced in [5,4]. It was shown that the mod- 
ular strategies problem is NP-complete in pushdown games with reachability and parity objectives 
in general [5,4]. The results of [5] also presents more refined complexity results in terms of the num- 
ber of exit nodes, showing that if every module has single exit, then the problem is polynomial for 
reachability objectives [5] and in NP n coNP for parity objectives [4]. 

Our contributions. In this work we present a complete characterization of the computational and 
strategy complexity of pushdown games and pushdown systems (one-player pushdown games or push- 
down automata) with mean-payoff objectives. Solving a pushdown system (resp. pushdown game) 
with respect to a mean-payoff objective is to decide whether there exists a path that (resp. a winning 
strategy to ensure that every path possible given the strategy) satisfies the mean-payoff objective. 
Our main results for computational complexity are as follows. 

1. Global strategies. We show that pushdown systems (one-player pushdown games) with mean-payoff 
objectives under global strategies can be solved in polynomial time, whereas solving pushdown 
games with mean-payoff objectives under global strategies is undccidable. 

2. Modular strategies. Solving pushdown systems with single exit nodes with mean-payoff objectives 
under modular strategies is NP-hard, and pushdown games with mean-payoff objectives under 
modular strategies can be solved in NP. Thus both pushdown systems and pushdown games with 
mean-payoff objectives under modular strategies are NP-complete. 

Our results are shown in Table 1. First observe that our hardness result for modular strategies is 
different from the NP-hardness of [5] because the hardness result of [5] shows hardness for games 
with reachability objectives and require that the number of modules with multiple exit nodes are not 
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Global strategies 


Modular strategies 


Pushdown systems 


PTIME 


NP-complete 
(NP-hard for single exit) 


Pushdown games 


Undecidable 


NP-complete 



Table 1. Computational complexity of pushdown systems and pushdown games with mean-payoff objectives. 





Global strategies 


Modular strategies 


Pushdown systems 


Infinite 


Memoryless 


Pushdown games 


Infinite 


Memoryless 



Table 2. Strategy complexity of pushdown systems and pushdown games with mean-payoff objectives. 



bounded (in fact if every module of the recursive game has a single exit, then the problem is in PTIME 
for reachability and NP n coNP for parity objectives). In contrast we show that for mean-payoff 
objectives the problem is NP-hard even for pushdown systems (only one player), where every module 
has a single exit node, under modular strategies. Second we also observe the very different complexity 
of global and modular strategies for mean-payoff objectives in pushdown systems vs pushdown games: 
the global strategies problem is computationally inexpensive (in PTIME) as compared to the modular 
strategies problem (which is NP-complete) in pushdown systems; whereas the global strategies problem 
is computationally infeasible (undecidable) as compared to the modular strategies problem (which is 
NP-complete) in pushdown games. Also observe that in contrast to finite-state game graphs where 
the complexities for mean-payoff and parity objectives match, for pushdown systems and games, the 
complexities of parity and mean-payoff objectives are very different. Along with the computational 
complexities, we also establish the optimal strategy complexity showing that global winning strategies 
for mean-payoff objectives in general require infinite memory even in pushdown systems; whereas 
memoryless or positional (independent of history) strategies suffice for modular strategies for mean- 
payoff objectives in pushdown games (see Table 2). Finally we also study the stack boundedness 
conditions where the goal of one player along with maximizing the mean-payoff objectives is also to 
ensure that the height of the stack is bounded. We show that all the complexities for the additional 
stack boundedness condition along with mean-payoff objectives are the same in pushdown systems 
and games as without the stack boundedness condition. 

Technical contributions. Our key technical contributions are as follows. For pushdown systems un- 
der global strategies we show that the mean-payoff objective problem can be solved by only considering 
additional stack height that is polynomial. We then show that the stack height bounded problem can 
be solved in polynomial time using a dynamic programming style algorithm. For pushdown games un- 
der global strategies our undecidability result is obtained as a reduction from the universality problem 
of weighted automata (which is undecidable [26, 1]). For modular strategies we first show existence of 
a cycle independent modular strategies, and then show memoryless modular strategies are sufficient. 
Given memoryless modular strategies and polynomial time algorithm for pushdown systems, we ob- 
tain the NP upper bound for the modular strategies problem. Our NP-hardness result for modular 
strategies is a reduction from the 3-SAT problem. 

Organization. Our paper is organized as follows. In Section 2 (resp. Section 3) we present the results 
for pushdown systems (resp. pushdown games) under global strategies. In Section 4 we present the 
results for modular strategies. 

2 Mean-Payoff Pushdown Graphs 

In this section we consider pushdown graphs (or pushdown systems) with mean-payoff objectives. We 
start with the basic notion of stack alphabet and commands. 

Stack alphabet and commands. Let r denote a finite set of stack alphabet, and Com(_T) = {skip, pop}U 
{push(z) | z e r} denote the set of stack commands over r. Intuitively, the command skip does 
nothing, pop deletes the top element of the stack, push(z) puts z on the top of the stack. For a stack 
command com and a stack string a <E r + we denote by com (a) the stack string obtained by executing 
the command com on a. 
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Weighted pushdown systems. A weighted pushdown system (WPS) (or a weighted pushdown 
graph) is a tuple: 

A= {Q,T,q £Q,E C (Q x r) x (Q x Com(r)),io : £ -> Z), 

where Q is a hnite set of states with go as the initial state; -T the finite stack alphabet and we assume 
there is a special initial stack symbol ± e E: E describes the set of edges or transitions of the pushdown 
system; and w is a weight function that assigns integer weights to every edge. We assume that _L can 
be neither put nor removed from the stack. A configuration of a WPS is a pair (a, q) where a 6 E + 
is a stack string and q G Q. For a stack string a we denote by Top(a) the top symbol of the stack. 
The initial configuration of the WPS is (_L, go)- We use W to denote the maximal absolute weight of 
the edge weights. 

Successor configurations and runs. Given a WPS A, a configuration Cj+i = (a i+1 ,q i+ i) is a suc- 
cessor configuration of a configuration a = (cti,qi), if there is an edge (c?i,7i, tft+i, com) € E 
such that com(cti) = a i+ i, where 7$ = Top(aj). A path n is a sequence of configurations. A path 
7r = (ci, . . . , c n+ i) is a waZirf paift if for all 1 < i < n the configuration Cj+i is a successor configuration 
of Cj (and the notation is similar for infinite paths). In the sequel we shall refer only to valid paths. 
Let tt = (ci, c 2 , . . . , Cj+i, . . .} be a path. We denote by n\j] = Cj the j-the configuration of the path 
and by 12] — (c^ , c il+ i, . . . , c i2 ) the segment of the path from the ii-th to the 12-th configuration. 
A path can equivalently be defined as a sequence (cieie 2 . . . e n ), where c\ is the initial configuration 
and ej are valid transitions. 

Average weights of paths. For a finite path n, we denote by 10(71") the sum of the weights of the 
edges in tt and Avg(Tr) = where \tt\ is the length of 7r, denote the average of the weights. 

For an infinite path tt, we denote by LimSupAvg(7r) (resp. LimlnfAvg(7r)) the limit-sup (resp. limit- 
inf) of the averages (long-run average or mean-payoff objectives), i.e., limsup(Avg(7r[0, i]))i>o (resp. 
lim inf(Avg(7r[0, i]))i>o). 

Notations. We shall use (i) 7 or 7$ for an element of E; (ii) e or ej for a transition (equivalently an 
edge) from E; (hi) a or on for a string from E* . For a path 7r = (ci, C2, . . .) = (cieie2 • • •) we denote 
by (i) qf. the state of configuration a, and (ii) on: the stack string of configuration a. 

Stack height and additional stack height of paths. For a path tt = ((«i, qi), ■ . ■ , (a n , q n )), the stack 
height of 7r is the maximal height of the stack in the path, i.e., SH(-7r) = max{|ai|, . . . , |a n |}. The 
additional stack height of 7r is the additional height of the stack in the segment of the path, i.e., the 
additional stack height ASH(7r) is SH(7r) — max{|ai|, |a„|}. 

Pumpable pair of paths. Let tt = {c\e\e2 ■ ■ ■) be a finite or infinite path. A pumpable pair of paths 
for 7r is a pair of non-empty sequence of edges: (j>i,P2) = (e^e^+i . . . ej l + ni , ei 2 ei 2 +i . . . ei 2 + n2 ), for 
n\,U2 > 0, i\ > and i^ > i\ + ri\ such that for every j > the path 7rj? pi obtained by pumping 
the pair pi and p 2 of paths j times each is a valid path, i.e., for every j > we have 

n (pi,P2) = ( c i e i e 2 • ■ • e il -i{e n e il+1 . . . e il+in ) 3 e ll+ln+1 . . . e l2 -i(e l2 e i2+ i . . . e J2+ „ 2 ) J e i2+ „ 2 . . .} 

is a valid path. We will show that large additional stack height implies the existence of pumpable pair 
of paths. To prove the results we need the notion of local minima of paths. 

Local minima of a path. Let 7r = (c±, C2, . . .) be a path. A configuration Cj = (ctj, qi) is a local minima if 
for every j > i we have on C (i.e., the stack string Qj is a prefix string of ctj). One basic fact about 
local minima of a path is as follows: Every infinite path has infinitely many local minimas. We discuss 
the proof of the basic fact and some properties of local minima. Consider a path tt = (ci,c 2 , . . .}. 
If there is a finite integer j such that from some point on (say after i-th index) the stack height is 
always at least j, and the stack height is j infinitely often, then every configuration after z-th index 
with stack height j is a local minima (and there are infinitely many of them). Otherwise, for every 
integer j, there exists an index i, such that for every index after i the stack height exceeds j, and 
then for every j, the last configuration with stack height j is a local minima and we have infinitely 
many local minimas. This shows the basic fact about infinitely many local minimas of a path. We 
now discuss a property of consecutive local minimas in a path. If we consider a path and the sequence 
of local minimas, and let a and Cj be two consecutive local minimas. Then either a and Cj have the 
same stack height, or else Cj is obtained from a with one push operation. In the following lemma we 
establish that if the additional stack height of a path exceeds (|Q| • |-T|) 2 , then there is a pumpable 
pair of paths. 
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Fig. 1. Subpath construction. 



Lemma 1 Let n be a finite path such that ASH(-7r) = d > (\Q\ ■ \r\) 2 . Then n has a pumpable pair of 
paths. 

Proof. We first select a subpath of it* of 7r such that 7r* = (c*, . . . , c*, . . . , c*) and the following 
conditions hold: (i) c* is a local minima in 7r*, (ii) \a*\ — |a* |, and (hi) |a*| = |a*| + d. The subpath 
is selected as follows: consider the configuration c* in 7r where the stack height is maximum, and c\ 
is the closest configuration before c* where the stack height is exactly d less than the stack height of 
c*, and similarly c* is the closest configuration after c* where the stack height is exactly d less than 
that of c* (see Figure 1). Clearly all the three conditions are satisfied. Let c\. (rcsp. c'(.) be the closest 
configuration before (resp. after) c* such that the stack height of c\. (resp. c".) is \a\ \ + j, for j > 0. 
Since d > (\Q\ ■ |-T|) 2 it follows from the pigeonhole principle that there exists j\,ji such that the 
states and the top stack symbol of and are identical, and the states and the top stack symbol 
of d( and c" are identical. It is straight forward to verify that the sequence p\ of edges between 
and along with the sequence p 2 of edges between d( and d( are a pumpable pair, i.e., (pi,p 2 ) 
is a pumpable pair for 7t. □ 

In the following lemma we establish the connection of additional stack height and existence of 
pumpable pair of paths with positive weights. 

Lemma 2 Let ci,c 2 be two configurations and n e Z. Let d E N be the minimal additional stack 
height of all the paths between C\ and c 2 with total weight at least n. If d > (\Q\ • |-T|) 2 , then there 
exists a path n* from c\ to c 2 with additional stack height d that has a pumpable pair (pi,p 2 ) such 
that w(pi) + w(p 2 ) > 0. 

Proof. The proof is by induction on the length \tt\ of the witness path 7r (i.e., 7r is a path from c\ to 
c 2 with minimal additional stack height with weight at least n) . We prove the base case and inductive 
step below. 

— Base case: Note that since ASH(7r) = d we must have \tt\ > d. Hence in the base case we have 
7r| = d. By Lemma 1, the path tt has a pumpable pair (pi,p 2 ). As |7r| — d it must be that the 
additional stack height ASH(7r| ) pi of the path 7r° pi p ^ obtained from 7r by removing pi and 
Pi (i.e., pumping (pi,p 2 ) zero times) is less than d. Hence it must be that w(ir) > w(ir9 

as otherwise ir® pi p ^ would be a witness path from c\ to c 2 with weight at least n but smaller 
additional stack height contradicting the minimality of it. Hence we have that w(pi) + w(p 2 ) > 0. 

— Inductive step: Again by Lemma 1, we have that ir has a pumpable pair (pi,p 2 ). If w(p\) + w{p2) > 
0, then we are done. Otherwise, w{pi) + w(p 2 ) < 0, and then we have w(ir9 > w(ir). Hence 
ASH(7r? is d, since, otherwise if ASH(7r? < d, then 7r? ^ would be a witness path 

V \Pl?P2)> ' ' v (Pl,P2)' ' (Pl,P2) 1 

from ci to c 2 with weight at least n but smaller additional stack height than 7r (contradicting 
minimality of ir). Since |7r| > |7r^ pi p J, by the inductive hypothesis it follows that 7r° pi p ^ has a 
pumpable pair (p3,_p/t) such that w(ps) + w(p±) > 0. 

The desired result follows □ 



Mean-payoff objectives with strict and non-strict inequalities. For a given integer r, the mean- 
payoff objective LimlnfAvg m r (resp. LimSupAvg x r) defines the set of infinite paths tt such that 
LimlnfAvg(7r) tx r (resp. LimSupAvg(7r) ix r), where ixs {>, >}. The mean-payoff objectives with inte- 
ger threshold r can be transformed to threshold by subtracting r from all transition weights. Hence in 
this work w.l.o.g we will consider mean-payoff objectives (i) LimlnfAvg > (resp. LimSupAvg > 0), and 
call them mean-payoff objectives with strict inequality; and (ii) LimlnfAvg > (resp. LimSupAvg > 0), 
and call them mean-payoff objectives with non-strict inequality. We are interested in solving WPSs 
with mean- payoff objectives, i.e., to decide if there is a path that satisfies the objective. 

2.1 Objectives LimlnfAvg > and LimSupAvg > 

In this section we consider limit-average (or mean-payoff) objectives with strict inequality. We will 
show that WPSs with such objectives can be solved in polynomial time. A crucial concept in the proof 
is the notion of good cycles, and we define them below. 

Good cycle. A finite path tt = (a,..., c n ) is a good cycle if the following conditions hold: 

1. w(ir) > (the weight of the path is positive); 

2. ci is a local minima; 

3. let ci = (ai,gi) and c„ = (a n ,q n ), then q x = q n and Top(ai) = Top(a„). 

We first prove two lemmas and the intuitive descriptions of them are as follows: In the first lemma 
we show that for every WPS, for every natural number d there exists a natural number n such that if 
there is a path with weight at least n and additional stack height at most d, then there is a good cycle 
in the WPS. The second lemma is similar to the first lemma, and shows that if the additional stack 
height of the path is large, then it is possible to construct paths with arbitrarily large weights. Using 
the above two lemmas we then show that if the weight of a finite path is sufficiently large, then either 
a good cycle exists or paths with arbitrarily large weights can be constructed. Finally we prove the 
key lemma that establishes the equivalence of the existence of a path satisfying mean-payoff objectives 
with strict inequality and the existence of a good cycle. 

Lemma 3 Let A be a WPS. For every dei there exists n^d £ N such that the following assertion 
holds: If there exists a path tt = (ci, . . . ,c r ) such that (i) c\ is a local minima, (ii) w(tt) > n^d and 
(in) ASH(7r) < d; then A has a reachable good cycle. 

Proof. Let Gd be a graph that contains all the paths that begin in c\ and end in c r with additional 
stack height at most d and for which c\ is a local minima. Clearly, the graph Gd is a finite graph, and 
the maximal absolute weight W is the same as that for A. A reachable positive cycle in Gd implies 
the existence of a reachable good cycle in A, and if no positive cycle is reachable, then the weight of 
each path is bounded by \Gd\ ■ W. Thus with n^^d = \Gd\ • W we obtain the desired result. □ 

Lemma 4 Let A be a WPS. Let n £ Z and let tt = (ci, . . . , c r ) be a path with weight at least n, with 
minimal additional stack height, such that C\ is a local minima. If ASH (tt) > (\Q\ ■ \r\) 2 , then for 
every m e N there exists a path n m from c\ to c r such that c\ is a local minima and w{-K m ) > m. 

Proof. By Lemma 2 there exists a path W from ci to c r that has a pumpable pair (pi,p 2 ) such that 
w (Pi) + w(p2) > 0. Hence for every i s N we get that w(tt 1 ^ >) > w(t^ Pi p ^) (i.e., the weight 
after pumping i+1 times the pair of paths exceeds than the weight of pumping i times). Hence for 
i = m — w(tt) we get that w(tT^ Pi n) > m. The desired result follows. □ 

Lemma 5 Let A be a WPS. There exists rij, € N such that if there exists a path tt from configuration 
c\ to configuration c r with c\ as a local minima and w{tt) > nj,, then one of the following holds: 

1. The WPS A has a reachable good cycle. 

2. For every n' G N there exists a path tt' from C\ to c r such that C\ is a local minima of tt' and 
w(tt') > n' . 

Proof. Observe that the is our choice and we will choose it sufficiently large for the proof. Let 
d* = {\Q\ ■ \r\) 2 , and our choice of is \Q\ ■ \r\ ■ n^d* (where n^d* is as defined in Lemma 3). Let 
tt = (a, C2, . . . , c r ) be a path such that c\ is a local minima and w(tt) > nj±. Let mi, . . . ,rrij be the 
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local minimas along the path. Note that mi = c\ and c r = rrij. Also note that j — \a r \ — \oti\- Note 
that if m il = (a il7 q) and m; 2 = (ctij 2 ,g), then if a good cycle does not exist we get that the weight 
of the path between m il and m; 2 is non positive. Hence, since Q and r are finite, by the pigeonhole 
principle, either a good cycle exists or there exists rrii, m i+ i such that aj+i = for some 7 € TU {e} 
(where e denotes the empty string) and there exists a path from rrn to m, + i such that m t is a local 
minima and the weight of the path is at least n^d* ■ Let tt* be such a path with minimal additional 
stack height between m t and m, + i. We consider two cases to complete the proof. 

1. If the additional stack height of tt* is smaller than d* , then by Lemma 3 we have a reachable good 
cycle from rrii and since m, is reachable from c\ we have reachable good cycle from c\ (condition 1 
of the lemma holds). 

2. If the additional stack of tt* is at least d* , then by Lemma 4 for every no we can construct a path 
7r„ between rrii and m,+i with weight u>(7r„ ) at least no, and mi is a local minima of 7r„ . For 
n' £ N, let n = n' + W ■ \ir\, and let 7r' be the path constructed using the segment from c\ to 
rrii, then the path ir no , and then the segment of 7r from m^+i to c r . The configuration ci is a local 
minima of 7r' and the weight of tt' is at least n — W ■ \tt\ > n'. Hence it follows that for every n' we 
can construct a path from c\ to c r with c\ as a local minima and weight at least n' (condition 2 
of the lemma holds). 

This completes the proof of the lemma. □ 

Lemma 6 Let A be WPS. The following statements are equivalent: (i) There exists a path 7Ti with 
LimSupAvg(7Ti) > 0; (ii) there exists a path 7T2 with LimlnfAvg(7r 2 ) > 0; and (Hi) there exists a path tt 
that contains a good cycle. 

Proof. The direction from right to left is immediate. Let tt — tt\tt2 be a finite path in A such that tti is 
a good cycle. Let tt\ = c\e\e\ . . . e„ and TT2 — C2e\e\ . . . e„ 2 . The infinite path tt' = TT\C2{e\e\ . . . e^ 2 )" 
obtained by repeating the good cycle forever is a valid path which satisfies that LimSupAvg(7r') > 
LimlnfAvg(Tr') > 0. 

In order to prove the opposite direction, we consider an infinite path tt such that LimSupAvg(7r) > 0. 
Let q £ Q and 7 £ T be such that the sequence mi = (a il , q), m 2 = {a i2 , q), . . . is an infinite sequence 
of local minima of tt and Top(ai j ) = 7 (note that such state and symbol are guaranteed to exist due 
to the existence of infinitely many local minimas and finiteness of \Q\ and \r\). If there exists j > 1 
such that w(Tr[ii,ij]) > then by definition 7r[ii,ij] is a good cycle and the result follows. Otherwise 
let us assume that for every j > 1 we have w(n[ii,ij]) < 0. As LimSupAvg(7r) > it follows that for 
every n £ N there exists i n > 1 such that the path 7r[ii,i n ] contains a prefix with weight at least n 
(otherwise LimSupAvg(7r) < 0). We now use Lemma 5 to complete the proof. Let n = (where nj, 
is as used in Lemma 5). Let tt' = mi, . . . , c* be the prefix of 7r[ii,i n ] such that w(tt') > n. If the first 
condition of Lemma 5 holds (i.e., A has a good cycle), then we are done with the proof. Otherwise, 
by condition 2 of Lemma 5 it follows that for every uq £ N there exists a path 7r„ from mi to c* 
such that mi is a local minima and w(TT na ) > n . Let us choose n — W ■ \ir[ii, i n ]\ + 1. Then consider 
the path W — TT no Tr[i + \Tr'\,i n ) that is obtained by concatenating the witness path 7r„ for n from 
mi to c* , and then the part of tt from c* to 7r[i n ]. For the path W we have (i) the sum of weights is 
at least no ~ W ■ |7r[zi,i n ]| > 1 > 0; (ii) 7r[ii] is a local minima; and (iii) the state and the top stack 
symbol of n[ii] and Tr[i n ] are the same. Thus n is a witness good cycle. For conclusion we get that if 
LimSupAvg(-7r) > 0, then there exists a good cycle, which also implies that there exists a path tt' such 
that LimlnfAvg(7r') > 0. This concludes the proof of the lemma. □ 

In the above key lemma we have established the equivalence of the decision problems for WPSs 
with mean-payoff objectives with strict inequality and the problem of determining good cycles. We 
will now present a polynomial time algorithm for detecting good cycles. To this end we introduce the 
notion of non-decreasing paths and summary functions. 

Non-decreasing paths. A path from configuration (cry, 91) to configuration (crfa^,^) is a non- 
decreasing a-path if (0:7,(71) is a local minima. Note that if tt is a non-decreasing a-path for some 
a £ r*, then it is a non-decreasing /3-path for every /3 £ T* . Hence we say that tt is a non-decreasing 
path if there exists a £ T* such that tt is a non-decreasing a-path. 

Summary function. Let A be a WPS. For a £ T* we define s a : Q x _T x Q — > { — 00} U Z U {uj} as 
following. 
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1. s a (<Zi,7>92) = w iff for every n G N there exists a non-decreasing path from (07,(71) to (07,(72) 
with weight at least n. 

2. s a (qi, 7, (72) = 2 G Z iff the weight of the maximum weight non-decreasing path from configuration 
(07,(71) to configuration (07,(72) is z. 

3. s Q ((7i, 7, (72) = —00 iff there is no non-decreasing path from (07,(71) to (07,(72). 

Remark 1 For every ai,a 2 G r* : s ai = s a . 2 . 

Due to Remark 1 it is enough to consider only s = s±. The computation of the summary function 
will be achieved by considering stack height bounded summary functions defined below. 
Stack height bounded summary function. For every d G N, the stack height bounded summary function 
Sd ■ Q x r x Q — > { — oc} U Z U {uj} is defined as follows: (i) s d (qi, 7, (72) = w iff for every n G N there 
exists a non-decreasing path from (±7,(71) to (±7,(72) with weight at least n and additional stack 
height at most d; (ii) Sd(qi,j,q 2 ) = z iff the weight of the maximum weight non-decreasing path from 
(J-7, qi) to (±7,(72) with additional stack height at most d is z; and (iii) Sd(<7i, 7, (72) = — 00 iff there 
is no non-decreasing path with additional stack height at most d from (±7,(71) to (±7,(72). 
Basic facts of summary functions. We have the following basic facts: (i) for every d G N, we have 
Sd+i > s <i (monotonicity); and (ii) Sd+i is computable in polynomial time from Sd and .4 (we will 
show this fact in Lemma 9). We first present a lemma that shows that from Sd, with d — {\Q\ ■ |r|) 2 , 
we obtain the values of function s for all values in Z U {—00}. 

Lemma 7 Let d = {\Q\ ■ \r\) 2 . For all (71,(72 G Q and 7 G r, if s(gi,7, 52) G Z U {—00}, then 
s{qi,l,l2) = Sd(qi,7,Q2)- 

Proof. By definition we have 5(91,7,(72) > Sd(qi, 7, (72). Assume towards contradiction that 
s (?ii 7j 92) > Sd(qi, 7, (72), then there exists a non-decreasing path 7r with minimal additional 
stack height from (±7,(71) to (±7,(72) with weight n > Sd(qi,j,q 2 ) and additional stack height 
d' > (\Q\ ■ l-H) 2 - Hence by Lemma 4 for every m G N there exists a non-decreasing path from (±7, q\) 
to (±7,(72) with weight at least m (note that in Lemma 4 the witness path constructed by pumping 
the positive pumpable pair yields a non-decreasing path). Hence 5(91,7,(72) = uj in contradiction to 
the assumption that s(qi,-f,q 2 ) G Z U {—00}. The desired result follows. □ 

Our goal now is the computation of the uj values of the summary function. To achieve the compu- 
tation of uj values we will define another summary function s* and a new WPS A* such that certain 
cycles in .4* will characterize the uj values of the summary function. We now define the summary 
function s* and the pushdown system .4*. Let d = (\Q\ ■ \r\) 2 . The new summary function s* is 
defined as follows: if the values of Sd and Sd+i are the same then it is assigned the value of Sd, and 
otherwise the value uj. Formally, 



The new WPS A* is constructed from A by adding the following set of w-edges: {(91,7,(72, skip) | 
s* (qi, 1, 92) = oj}. 

Lemma 8 For all (71, (72 G Q and 7 G r, the following assertion holds: the original summary function 
s(qi, 7, (72) = uj iff there exists a non-decreasing path in A* from (±7, qi) to (±7, q 2 ) that goes through 
an uj-edge. 

Proof. The direction from right to left is easy: if there is a non-decreasing path in .4* that goes 
through an w-edge, it means that there exists (<7i,7', q' 2 ) with either Sd{q[, 7, q' 2 ) = uj or s,i(g^ , 7', q' 2 ) < 
Sd+iiq'i,!',^)- If s d (q' 1 ,l,q' 2 ) = oj, then clearly s(q[,^,q' 2 ) = uj. Otherwise we have s d (q[,-f' ,q' 2 ) < 
s d+ i(q' 1 , 7', q' 2 ), and then the proof of Lemma 7 shows that s(q[,j' ,q' 2 ) = uj. Since there exists finite 
path from (±7, qi) to (±7, q 2 ) with the uj-edge it follows that s(qi, 7, q 2 ) = uj. 

For the converse direction, we consider the case that s(qi, 7,(72) = UJ- If s* (qi, 7, q 2 ) = uj, then 
the proof follows immediately. Otherwise it follows that Sd{qi, 7, q 2 ) G Z. Hence there exists a weight 
n G Z such that the non-decreasing path with the minimal additional stack height with weight n 
has additional stack height d! > d + 1. Let n be that path. Then there exists a non-decreasing 
subpath that starts at (07',^) and ends at (aj',q' 2 ) with additional stack height exactly d+ 1. If 
Sd+i(q[, 7', q 2 ) = Sd(q[,j', q 2 ), then ir is not the path with the minimal additional stack height. Hence, 
as Sd+i(q' l7 7', q 2 ) > Sd(q' l7 j' 7 q' 2 ), by definition s*(q[, 7', q 2 ) = uj and the proof follows. □ 




Sd{qi,l,q2) if s d {q ul ,q 2 ) = s d +i{qi,^,q 2 ) 
uj if s d ( qi ,j,q 2 ) < s d+ i(qi,J,q 2 ). 
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We are now ready to show that the summary function s can be computed polynomial time. 

Lemma 9 For a WPS A, the summary function s is computable in polynomial time. 

Proof. There are two key steps of the proof: (i) computation of Sd, for d = (\Q\ ■ \r\) 2 , and we will 
argue how to compute Sj+i from in polynomial time; (ii) computation of a non-decreasing path in 
A* that goes through an cj-edge. We first argue how the key steps give us the desired result and then 
present the details of the key steps. Given the computation of (i), we construct Sd, Sd+i in polynomial 
time, and hence also s*. Given s* we construct A* in polynomial time. By computation (ii) we can 
assign the uj values for the summary function, and all other have values as defined by s<j- Thus with 
the computation of key steps (i) and (ii) in polynomial time, we can compute the summary function 
s in polynomial time. We now describe the key steps: 

1. Computation of s i+ i from s, and A. Let Ga be the finite weighted graph that is formed by all 
the configurations of A with stack height either one or two, that is, the vertices are of the form 
(a, q) where q £ Q and a £ {_L • 7, _L • 71 • 72 | 7, 71, 72 £ r}. The edges (and their weights) are 
according to the transitions of A: formally, (i) (Skip edges): for vertices (_L • a, q) we have an edge 
to (_L • a, q') iff e = (q, Top(a), skip, q') is an edge in A (and the weight of the edge in Ga is w(e)) 
where a = 7 or a = 71 • 72 for 7,71,72 £ -T; (ii) (Push edges): for vertices (± ■ 7,(7) we have an 
edge to (_L • 7 • 7', q') iff e = (q, 7, push(Y), q') is an edge in A (and the weight of the edge in Ga 
is w(e)) for 7, 7' £ r; and (iii) (Pop edges): for vertices (_L • 7 ■ 7', q) we have an edge to (_L • 7, q') 
iff e = (q,-f',pop,q') is an edge in A (and the weight of the edge in Ga is w(e)) for 7,7' £ r. 
Intuitively, Ga allows skips, push pop pairs, and only one additional push. Note that Ga has at 
most 2 • \Q\ ■ \r\ 2 vertices, and can be constructed in polynomial time. 

For every i > 1, given the function Sj, the graph G\ is constructed from Ga as follows: adding 
edges ((T7172, gi), (-L7172, (72)) (if the edge does not exist already) and changing its weight to 
Sj(<?i, 72, (72) for every 71,72 £ r and (71,92 £ Q- The value of Si + i(q\, 7, 92) is exactly the weight 
of the maximum weight path between (±7, qi) and (±7, q 2 ) in G\ (with the following convention: 
— 00 < z < uj, z + uj — uj and z H — 00 = to H — 00 = —00 for every z £ Z). If in G\ there is a path 
from (_l_7, qi) to (±7, q 2 ) that contains a cycle with positive weight, then we set s i+ i(qi, 7, q 2 ) = w. 
Hence, given Sj and A, the construction of G\ is achieved in polynomial time, and the computation 
of Sj+i is achieved using the Bellman-Ford algorithm [13] in polynomial time (the maximum weight 
path is the shortest weight if we define the edge length as the negative of the edge weight). Also 
note that the Bellman-Ford algorithm reports cycles with positive weight (that is, negative length) 
which is required to set oj values of Sj+i. It follows that we can compute s i+ i given and A in 
polynomial time. 

2. Non-decreasing uj-edge path in A*. We reduce the problem of checking if there exists a non- 
decreasing path from (-L7, q{) to (±7,(72) in A* that goes through an w-edge to the problem 
of pushdown reachability in pushdown systems (or pushdown graphs), which is known to be in 
PTIME [34, 2]. The reduction is as follows: for every state q £ Q we add a fresh (new) state q" , add 
a transition (or edge) (qf,^, q 2 , com) for every ((71,7,(72, com) £ A (i.e., the freshly added states 
follow the transition in the fresh copy as in the original WPS), and a transition (91,7, q 2 , com) 
for every transition (gi, 7, q 2 , com) that has an uj weight (i.e., there is a transition to the fresh 
copy only for an w-edge). It follows that there exists an w-edge non-decreasing path in A* from 
(±7,(71) to (±7,(72) iff the configuration (±j,q 2 ) is pushdown reachable from the configuration 
(J_7, <7i). Hence it follows that existence of non-decreasing w-edge path in A* can be determined 
in polynomial time. 

This completes the proof of polynomial time algorithm to compute the summary function. □ 

Given the computation of the summary function, we will construct a summary graph, and show 
the equivalence of the existence of good cycles in a WPS with the existence of positive cycles in the 
summary graph. 

Summary graph and positive simple cycles. Given a WPS A = (Q,T,qo £ Q, E C (Q x r) x (Q x 
Com(/ n )), w : E — > Z) and the summary function s, we construct the summary graph Gr(^l) = (V, E) 
of A with a weight function w : E ZU{uj} as follows: (i) V = QxT; and (ii) E = E s ki V VJE vus h where 
Eskiy = {({qi, 7), (Q2, 7)) I s(q!,j,q 2 ) > -00}, and E push = {{{q t , 71), (q 2 , 72)) | {q-i,ji,q 2 ,push(-f 2 )) £ 
E}; and (iii) for all e = ((171,7), ((72,7)) € E sklp we have w(e) = s(q 1 , 7, 172), and for all e £ E push we 
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have w(e) = w(e) (i.e., according to weight function of A). A simple cycle C in Gr(A) is a positive 
simple cycle iff one of the following conditions hold: (i) either C contains an w-edge (i.e., edge labeled 
uj by w); or (ii) the sum of the weights of the edges of the cycles according to w is positive. 

Lemma 10 A WPS A has a good cycle iff the summary graph Gr(„4) has a positive simple cycle. 

Proof. If A has a good cycle, then let n be a good cycle. The good cycle tt is a non-decreasing path 
(ci, . . . , c„) such that c\ = (017, q) and c„ = (0:170:27, q) and w(ir) > 0. Let mi,...,m r be the local 
minimas along the path. Note that for every i < r, either m, and m,+i have the same stack height 
or mj+i is reachable from rrii via one push transition. For configuration c = (07,(7), let us denote 
Top(c) = (7,(7). Hence the path Top(mi), . . . ,Top(m r ) is a cycle in Gr(„4). If the cycle contains an 
w-edge, then it is a positive cycle (by definition of positive cycles in Gr(_4)). Otherwise, the weight of 
the cycle in Qr{A) is at least w(w), and therefore Gr(A) has a positive cycle (and therefore positive 
simple cycle). 

The other direction is as follows. Consider a positive cycle in Gr(.4). If the cycle does not contain 
an cj-edge, then there exists a non-decreasing path in A with the same weight that forms a good cycle. 
Otherwise, let (7, q) be a vertex in the cycle, and ((71, 91), (71, 92)) be an w-edge in the cycle of Gr(^4). 
From the construction of Gr(A), it follows that there exist 01,02,0:3 in A such that the following 
non-decreasing paths exist: 

— A non-decreasing path 7Ti from (017, q) to (0170271,(71) (due to the path of the cycle). 

— For every m G N: a non-decreasing path 7r m from (0170271,^1) to (0170271,(72) with weight at 
least m (due to the w-edge). 

— A non-decreasing path tt 2 from (0170271,(72) to (017027103,(7) (due to the path of the cycle). 

Hence, for m — W ■ + |7r 2 |) + 1, we get that the path 7ri7r m 7r 2 is a good cycle. This completes 
both directions of the proof and gives us the result. □ 

Since the summary function and summary graph can be constructed in polynomial time, and the 
existence of a positive cycle in a graph can be checked in polynomial time (for example, first checking 
existence of a cycle with an w-edge, and then applying Karp's mean-cycle algorithm [25] after removing 
all co edges), we have the following lemma. 

Lemma 11 Given a WPS A, whether A has a good cycle can be decided in polynomial time. 

Lemma 6 and Lemma 11 give us the following theorem. 

Theorem 1 Given a WPS A, whether there exists an infinite path n such that LimlnfAvg(7r) > 
(or LimSupAvg(7r) > 0) can be decided in polynomial time. If there exists an infinite path n such 
that LimSupAvg(7r) > 7 then there exists an ultimately periodic infinite path n' such that both 
LimSupAvg(7r') > and LimlnfAvg(7r') > 0. 

2.2 Objectives LimlnfAvg > and LimSupAvg > 

In this section we consider mean-payoff objectives with non-strict inequality. We will assume that the 
input WPS A has integer weights, but we will consider certain transformations that produce rational 
weight functions. 

Transformed weight functions and weighted graphs. Let w : E — > Q be a weight function, and r £ Q 
be a rational value, then the weight function w + r : E — > <Q> is defined as follows: for all e £ E we 
have (w + r)(e) = w(e) + r. Let G = (V 7 E) be a (possibly infinite) 3 graph with a weight function 
w : E — > Q. In order to emphasize that w is the weight function for G, we use wg- We denote by G r 
the same infinite graph with weight function wq + r. We first show that if the lim-inf-average objective 
can be satisfied for all e > 0, then the non-strict lim-inf-average objective can also be satisfied. 

Lemma 12 Let A be a WPS. There exists a path n with LimlnfAvg(7r) > iff for every e > there 
exists a path 7r e with LimlnfAvg(7r e ) > — e. 

3 In this subsection we often look at a WPS as an infinite graph of the configurations 



10 



Proof. The direction from left to right is trivial. 

In order to prove the converse direction let us assume that for every n £ N there exists a path 7r„ 
with LimlnfAvg(7r„) > — i. Hence for every n £ N there exists a path tt* which leads to a path C n 
that is a good cycle with respect to the weight function w + ^. Since there are infinitely many values 
of n £ N, and Q and T are finite, w.l.o.g all the good cycles (with respect to w + ^) starts at the same 

top configuration (7,(7). Hence for tt = tt* Of ...Cf ' 4+11 ... we get that LimlnfAvg(Tr) > 0. 
This completes the proof. □ 

Lemma 13 Let A be a WPS with integer weights (weight function w). Let £ — \T\ ■ \Q\, and fix 
e = f {e+ ^-i ■ Then the WPS A e (with weight function w + e) has a good cycle iff for every S > the 
WPS A s (with weight function w + 5) has a good cycle. 

Proof. The direction from right to left is trivial. For the converse direction we first prove the following 
lemma. 

Lemma 14 Let s e be the summary function for A e . 
1- If s £ (qi,-y,q 2 ) ^ ui, then s e (q 1 ,j,q 2 ) < s(q 1 ,j,q 2 ) + jj. 

2. If s e (qi,7, q 2 ) — uj, then for every S > we have s d (qi, 7, g 2 ) — uj, where s 5 is the summary 
function for A 5 . 

Proof. We prove both the items below. 

1. If s e (qi,-y,q 2 ) ^ uj, then maximum weight non-decreasing path with minimal additional stack 
height from (-L7, gi) to (±-y,q 2 ) has an additional stack height of at most (|Q| • \r\) 2 = £ 2 . Note 
that this path does not contain positive cycles (since s e {q\, 7, q 2 ) ^ to). Hence there exists a path 
tt with the same weight and with stack height at most £ 2 which does not contain any cycles. Hence 
I -7r I < £ e , and therefore 

WA' (tt) = w a {k) + e ■ \w\ < w A {n) + e • l e < w a (tt) + —7^. 

Since s(q\, 7, q 2 ) > wa(tt) (as tt is a non-decreasing path we have s(qi, 7, q 2 ) > u>a{^)), we obtain 
the result of the first item. 

2. In order to prove the second item of the lemma, it is enough to prove that if an edge weight is 
uj in (A e )* (where (A e )* is the WPS constructed with the function (s e )*), then for every S > 
the weight of the edge is uj also in the summary graph Gr(A s ) of A s . We consider two cases to 
complete the proof. 

— Case 1. If s|(<7i, 7, q 2 ) = w , then the infinite graph A e has a positive cycle C with stack height 
at most £ 2 , hence there exists positive cycle C such that \C'\ < £ l . Towards contradiction, 
let us assume that wa(C) < 0. As all the weights in A are integers we get that wa(C) < — 1. 
As wa{C) + e ■ \C'\ = WA'(C') > we get that \C'\ > - which is a contradiction. Thus 
wa{C) > 0, and hence for every S > we have w^^(C") > 0, hence sf (qi, 7, q 2 ) = uj. 

— Case 2. Otherwise, we have s^ +1 (qi, 7, q 2 ) > s^(qi,j,q 2 ). Let tt be a path from (-L7, q\) to 
(±7,(72) with additional stack height £ + 1 and weight s| +1 (gi, 7, q 2 ). As s\ +1 {q\, 7, q 2 ) > 
s |(9i)7)92), by Lemma 2 it follows that tt has a pumpablc pair (pi,p 2 ) with u>a*{pi) + 
WA e {Pi) > 0. If pi (rcsp. p 2 ) contains a cycle with positive weight, then by the same ar- 
guments presented in the proof of the first item of the lemma this cycle will be positive also 
in A s , for every 6 > 0, and hence s s (qi, 7, q 2 ) = uj. Therefore w.l.o.g both p\ and p 2 do not 
contain any cycles and thus \p\\, \p 2 \ < £ . Again by the same arguments presented in the 
proof of the first item we obtain that wa(pi) + wa{p 2 ) > and hence for every i5 > we 
have w A s(pi) + w A s(p 2 ) > 0. As (pi,p 2 ) is a positive pumpable pair in A 5 it follows that 
s S {q\,l,q 2 ) = uj. 

This completes the proof of the second item. 

We obtain the desired result of the lemma. □ 

We are now ready to prove Lemma 13. Let us assume that there exists a good cycle in A e . Then 
by Lemma 10 there exists a positive simple cycle C in the summary graph Gr(.4 £ ). We consider two 
cases: 
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push(-y),— 1 pop(-y),l 




skip, — 1 



Fig. 2. WPS .4. to witness ultimately periodic words might not suffice for mean-payoff objectives with non- 
strict inequality 



— If C contains an w-edge e, then by Lemma 14 for every 5 > the same cycle in Gr(_4 5 ) will also 
contain an w-edge. Therefore C is a positive cycle also in Gr(A 5 ) and hence A s has a good cycle. 

— Otherwise C does not contain an w-edge. Towards contradiction assume that the weight of C in 
Gr(A) is negative. As the weight of A arc integers it follows that the weight of C is at most —1. 
By Lemma 14, for every e £ C we have WGr(>t' ! )( e ) < w Gr(.4)( e ) + 57; an d thus wq^^^C) < 
u 'Gr(^i)(C) + 27- As C is a simple cycle (in Gr(„4 e )) we get that |C| < £, and hence we have 
w Gr(A c )(C") ^ ^Grf^CC) + 5 < — which contradicts the assumption that C is a positive cycle. 
Therefore we have wq,^){C) > 0, and therefore for every 6 > we get that WQ r ^s^(C) > and 
hence A s has a good cycle. 

This completes the proof of the lemma. □ 

Theorem 2 Given a WPS A, whether there exists an infinite path tt such that LimlnfAvg(7r) > (or 
LimSupAvg(7r) > 0) can be decided in polynomial time. There exists a WPS A such that there exists a 
path tt with LimlnfAvg(7r) = but for every ultimately periodic path tt we have both LimlnfAvg(7r) < 
and LimSupAvg(7r) < 0. 

Proof. From Lemma 12 it follows that if there is a path tt such that LimlnfAvg(7r) > 0, then for every 
ei > there is a path tt such that LimlnfAvg(7r) > — e\. By Lemma 13 it follows that it suffices to 
check for e (for the e described by Lemma 13). Given a WPS A, the WPS A e can be constructed 
in polynomial time (as e has only polynomial number of bits). Then applying the polynomial time 
algorithm to find good cycles (as given in the previous subsection) we answer the decision problems 
in polynomial time. We observe that Lemma 12 and Lemma 13 also hold for LimSupAvg objectives, 
and thus the result also follows for LimSupAvg objectives. 

We now present the example to show that the witness paths for non-strict inequality mean-payoff 
objectives are not necessarily ultimately periodic. Consider the WPS A with two states Q — {qi, 92} 
with two symbol stack alphabet r = {_L, 7} and the edge set E = {ei, e2, . . . , 65} is described as fol- 
lows: ei = (q 1 ,±,q 1 ,push(-f)),e 2 = (qi,"f,qi,push(j)),e 3 = (q lt 7, q 2 , skip), e 4 = (q 2 ,J,q2,pop),e 5 = 
(q 2 , -L,qi, skip). The weight function is as follows: w{e±) — 1, and all other edge weights are — 1. 
(See Figure 2 for pictorial description). For i > 1, consider the path segment pi = eie 2 ~ 1 e 3 e\e5 
that executes the edge e\, followed by (i — l)-times the edge e 2 , then the edge e 3 , followed by i- 
times the edge e^ and finally the edge e$. It is straight forward to verify that for the infinite path 
7r = (±,qi)pip 2 p 3 ... we have that get that LimSupAvg(7r) = LimlnfAvg(7r) = 0. However for every 
valid path tt = (1^, where £1 £ E* and £ 2 £ E + it must be the case that cither (i) £ 2 = e 2 and 
then LimlnfAvg(7r) = LimSupAvg(7r) = —1 or that (ii) £2 is a cycle with length at most |£ 2 | has weight 
at most —1, and hence LimlnfAvg(7r) < LimSupAvg(7r) < — j^j < 0. This completes the proof of the 
result. □ 

2.3 Mean-payoff objectives with stack boundedness 

In this section we consider WPSs with mean-payoff objectives along with the stack boundedness con- 
dition that requires the height of the stack to be bounded. An infinite path n = (ci, c 2 , ... Cj . . .) is a 
stack bounded path if there exists n £ N such that \on\ < n for every i e M (recall that on is the stack 
string of configuration a). 

Theorem 3 Given a WPS A, the following problems can be solved in PTIME. 
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1. Does there exist a stack bounded infinite path tt such that LimlnfAvg(7r) cxi (resp. LimSupAvg(7r) x 

0), for M£ {>,>}? 

2. Is sup{LimlnfAvg(7r) j it is a stack bounded path} > (resp. 
sup{LimSupAvg(7r)|7r is a stack bounded path} > 0)? 

Proof. The results for each item are proved with a lemma of the proof below. 

Lemma 15 There exists a stack bounded infinite path it in A such that LimSupAvg(7r) > (resp. 
LimSupAvg(7r) > 0) iff the summary graph Gr(„4) has a vertex with self-loop that has a positive (resp. 
non-negative) weight. 

Proof. If there exists a stack bounded infinite path tt in A such that LimSupAvg(7r) > (resp. 
LimSupAvg(7r) > 0), then it contains a cycle that begins and ends at configuration (a-f,q) with 
positive (resp. non negative) weight. Hence in the summary graph Gr(_4) the vertex (7, q) will have a 
self-loop with positive (resp. non negative) weight. The other direction is straight forward. □ 

It is straight forward to verify that Lemma 15 also holds for LimlnfAvg(7r) objective. This gives us 
the first item of the theorem. The next lemma proves the last item of the theorem. 

Lemma 16 Let e be the constant from Lemma 13. Then there exists a stack bounded infinite path tt 
such that LimlnfAvg(-7r) > — e iff sup{LimlnfAvg(7r) | tt is a stack bounded path} > 0. 

Proof. The direction from right to left is immediate. In order to prove the other direction let us assume 
that there exists a stack bounded infinite path tt such that LimlnfAvg(7r) > — e. Hence by Lemma 15 
the summary graph Gr(.4 £ ) contains a self-loop for vertex (7, q) with positive weight. By the same 
argument used in the proof of Lemma 13 it follows that for every 6 > the self-loop of vertex (7, q) will 
have a positive weight in graph Gr(„4 5 ). Hence for every 5 > there exists a stack bounded path ng such 
that LimlnfAvg^) > —S, which implies that sup{LimlnfAvg(7r) | tt is a stack bounded path} > 0. □ 

The proof of Lemma 16 straight forwardly extends to LimSupAvg(7r) objective, and hence we have the 
desired result of the theorem. □ 

Thus we have the following result summarizing the computational complexity. 

Theorem 4 Given a WPS A, the following questions can be solved in polynomial time: (1) Whether 
there exists a path tt in x 0, where <P e {LimSupAvg, LimlnfAvg} and Me {>, >}; and (2) whether 
there exists a path tt in $ ix such that tt is stack bounded, where <P G {LimSupAvg, LimlnfAvg} and 
{>, >}. 

3 Mean-Payoff Pushdown Games 

In this section we consider pushdown games with mean-payoff objectives. We will show that the 
problem of deciding the existence of a strategy (or a finite-memory strategy) to ensure mean-payoff 
objectives in pushdown games is undecidable. The undecidability results will be obtained by a reduc- 
tion from the universality problem of weighted automata, which is known to be undecidable [26, 1]. 
We start with the definition of weighted pushdown games. 

Weighted pushdown games (WPGs). A weighted pushdown game (WPG) Q = {A, (Qi, Q2)) 
consists of a WPS A and a partition (Qi,Q2) of the state space Q of A into player-1 states Qi 
and player-2 states Q 2 - A WPG defines an infinite-state game graph (V, E) with partition {Vi,V 2 ) 
of the vertex set V, where V is the set of configurations of A, and V\ — {(a, q) e V | q € Qi}, 
V 2 = {(a,q) E V \ q E Q 2 } and E is obtained from the transitions of A. The initial vertex is the 
configuration (±,q ). 

Plays and strategies. A play on Q (or equivalently on the infinite-state game graph) is played in the 
following way: a pebble (or token) is placed on the initial vertex; and in every round, if the pebble 
is currently on player-1 vertex (a vertex in V\), then he chooses an edge to follow, and moves the 
pebble accordingly; and if the current vertex is a player-2 vertex, he does likewise. The process goes on 
forever and generates an infinite play (an infinite path tt in the infinite graph of the game) . A strategy 
for player 1 is a recipe to extend plays; formally, a strategy for player 1 is a function r : V* x V\ — > V 
such that for all w e V and v E V\ we have (v,t(w ■ v)) £ E. Equivalently a strategy for player 1 
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given a history of configurations (i.e., the sequence of configurations of the finite prefix of a play) 
ending in a player-1 state, chooses the successor configuration according to the transition of A. A 
play 7r = V1V2 ... is consistent with a strategy r if for every Vi € V\ we have Vj+i = t{v\V2 ■ ■ ■ 
i.e., the play is possible according to the strategy r. The definition of player-2 strategies is analogous. 
Informally a strategy can be viewed as a transducer that takes as input the sequence of transitions, 
and outputs the transitions to be taken. A strategy is finite-memory if there is a finite-state transducer 
to implement the strategy. 

Winning strategies. Wc will consider mean-payoff objectives, as already defined in the previous section. 
A player-1 strategy t is a winning strategy if for every play 7r consistent with r we have LimlnfAvg(7r) > 
(resp. LimlnfAvg(7r) > 0, LimSupAvg(7r) > 0, LimSupAvg(7r) > 0). In other words, a winning strategy 
for player 1 ensures the mean-payoff objective against all strategies of player 2. We are interested in 
the question of existence of a winning strategy, and the existence of a finite-memory winning strategy 
for player 1 in WPGs with mean-payoff objectives. Our undccidability results for WPGs with mean- 
payoff objectives will be obtained by reduction from the non-universality of weighted finite automata. 
We define the problem below. 

Weighted finite automata (WFA). A weighted finite automaton (WFA) is a tuple A = 
(E, Q, qo, A, w : A — > Z), where E is a finite input alphabet, Q is a finite set of states, A C Q x E x Q 
is a transition relation, w : A — > Z is a weight function and go € Q is the initial state. For a word 
p = o\02 ■ ■ ■ <r„, a run of A on p is a sequence r — r$ri . . . r n G Q + , where ro = qo, and for all 1 < i < n 
we have di — (rj_i, <7i, ri) G A. The weight of the run r is w(r) = Y^7=i w (^i)- Since the automaton 
is non-deterministic there maybe several runs for a word, and the weight of a finite word p G E* 
over A is the minimal weight over all runs on p, i.e., Lj((p) — min{w(r) | r is a run of A on p}. The 
non-universality problem asks, given v G Z, whether there exists a word p G E* for which Lj±{p) > ul 

Theorem 5 ([1]) Given a WFA with a weight function w : A — > {—1,0,1}, the non-universality 
problem is undecidable for every v G Z for the following question: Does there exist a word p G E* such 
that iyt(p) >v? (Equivalently, is it not the case that for every p G E* we have Lj,{p) < v — 1 ?) 

Informally, given a WFA A we will construct a WPG in such way that in the first rounds player-1 
fills the stack with letters that construct a word p of A, and then player-2 simulates the WFA minimal 
run on p and then the game returns to the initial state. If for all p G E* we have Lj({p) < 0, then the 
mean-payoff of the play will be at most 0, otherwise, there exists a word p G E* such that L^(p) > 0, 
and then by playing according to p, player-1 can ensure positive mean-payoff. 

Reduction: WFA to WPGs. Wc first prove that WPGs are undecidable for LimlnfAvg(7r) > 
and LimSupAvg(7r) > objectives. This proof will immediately show the undccidability also for 
LimlnfAvg(7r) > and LimSupAvg(7r) > objectives, as LimlnfAvg(7r) > (resp. LimSupAvg(7r) > 0) 
is dual to the objective of player 2 when the objective of player 1 is LimSupAvg(7r) > (resp. 
LimlnfAvg(Tr) > 0). 

Reduction. The reduction from the non-universality problem of a weighted automaton is as follows. 
Given a WFA A = (E,Q,q ,A,w : A -)■ {-1,0,1}} we construct a WPG Q with the aid of five 
gadgets, and we describe the gadgets below. WLOG we assume that there is a special symbol $ that 
does not belong to E. 

1. Gadget 1. The first gadget contains only one state, namely q$, which is a player-1 state. The state 
has two possible transitions. In the first transition it pushes $ into the stack and remains in the 
same state. In the second transition it pushes $ and goes to the second gadget. All the weights in 
this gadget are —10. 

2. Gadget 2. The second gadget also contains one state, namely qs, which is also a player-1 state. 
For every a G E the state has a transition that pushes a into the stack and remains in the same 
state. In addition there is one more transition, which leads to the third gadget keeping the stack 
unchanged with weight 0. All the weights in this gadget (other than the skip transition) are — 1. 
Informally, in this gadget player 1 needs to construct a word p such that the reverse of p has value 
at least 1 in A. For a word p, let rev(p) denote the reverse of the word. 

— In this gadget player 1 should construct a word p G E* for which L^(rev(p)) > 1. 

— The WPG Q will be constructed in such way that player 1 must play in a way so that the 
number of $ in the stack will be greater than the number of letters from E to ensure the 
mean-payoff objectives. 
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Fig. 3. The WPG Q from WFA A 



3. Gadget 3. The third gadget is the choice gadget with only one player-2 state q c t, which either 
leads to the fourth gadget or the fifth gadget. The weights of the transitions are and the stack 
is not changed. Informally, player-2 should go to the fifth gadget if the word that player 1 pushed 
into the stack has non-positive weight, and should go to the fourth gadget if the number of $ 
symbols in the stack is less than the number of symbols from S. 

4. Gadget 4- The fourth gadget consists of only one player-2 state q K $ (to denote that there is not 
enough $ symbols). It has a transition pop(a) with weight, for all a G S; and a transition pop($) 
with +11 weight. If the stack is empty, then there is a transition to the initial state. 

A pictorial description of the first four gadgets is shown in Figure 3, where O denote player- 
1 states, □ denote player-2 states, and edges are labeled by stack top; followed by the stack 
command; and then the weight; and if the stack top is irrelevant (i.e., the transition is valid for 
all stack tops), then it is denoted as _. We now describe the fifth gadget. 

5. Gadget 5. The fifth gadget is the simulate run gadget. The states in this gadgets are essentially 
the set Q of states of the automaton A; and all the states are player-2 states. The transitions 
and edge weights are as follows: (i) for every (q,cr,q') G A we have a transition (q,a,q' ,pop(a)), 
with weight w^q, a, q') + 1 (1 plus the weight in A); and (ii) in addition there exists a transition 
(q, $, q, pop($)) with weight +10 and a transition (q, _L, q$, skip) to the initial state for empty stack 
with weight 0. 

Correctness of reduction. We will now prove the correctness of the reduction by showing that 
there is a winning strategy (also a finite-memory winning strategy) in the WPG Q for mean-payoff 
objectives with strict inequality iff there is a finite word p G S* such that L^(p) > 1- Let tt be a play 
on the above WPG Q. The i-th iteration of the play are the rounds between the i-th visit and the 
(i + l)-th visit to the initial state. 

Lemma 17 // there is a word p G U* such that L^(p) > 1, then there exists a finite-memory strategy 
Tj* for player 1 to ensure that for all plays tt consistent with t* we have LimSupAvg(7r) > and 
LimlnfAvg(7r) > 0. 

Proof. The finite-memory strategy r* for player 1 is to play in every iteration in q$ followed by 
rev(,o) in qs, where n = \p\ is the length of the word p. In every iteration, the sum of the weights is 
at least 1 as L_^(p) > 1, and the length of play in every iteration is at most 4 • n. It follows that for 
all plays n consistent with T-j* we have both LimSupAvg(7r) > and LimlnfAvg(7r) > 0. □ 

Lemma 18 If for all words p G S* we have L_^(p) < 0, then there exists a counter strategy for 
player 2 to ensure that for all strategies t\ of player 1, for the play given t| and t\: for all iterations 
i, for every round between i-th iteration and (i + l)-th iteration, the sum of the weights from the 
beginning of the iteration to the current round of the iteration is at most 0. 
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Proof. The counter strategy t| is as follows: consider an iteration i, and let the strategy of player 1 
in this iteration produce the sequence $ n p, for pel". Note that if the state q c t is never reached, 
then all the weights are negative (in q$ and qs all weights are negative). The strategy is described 
considering the following two cases. 

1. If n < \p\, then the strategy r| chooses the state g<$ at the state q c h (since there are not enough $ 
in the stack). For any round of the play till state q c h is reached, the sum of the weights is negative. 
Once g<$ is reached, for any round the payoff is at most — 10 • n — \p\ + 11 • n = — \p\ + n < 0, as 
n < \p\. 

2. Otherwise, we have n > \p\ and L_^(rev(p)) < 0. There exists a run r on rev(p) such that for every 
prefix f3 of rev(p) the sum of the weights is at most 2-|/3|<2-|p|<2-n (since the absolute value 
of the weights of A are bounded by 1) and in the end of the run sum of the weights is at most 
0. The counter strategy r| follows the run r. Hence the sum of the weights for any prefix /3 is at 
most —10 • n + 2 • |/3 < —10 • n + 2 • n < 0, until the letter $ is the top symbol of the stack. Once 
$ is the top symbol, the sum of the weights is at most — 10 • n, since the sum of the weights of 
the run is at most 0. Since with each pop of $ the weight is 10, and there are n pops, it follows 
that in every round of iteration the sum of the weights is at most 0. Finally, once the iteration is 
completed the sum of the weights is also at most 0. 

The desired result follows. □ 

Lemma 19 Given WFA A and the WPG Q constructed by the reduction the following the assertions 
hold: 

1. If there is a word p G E* such that iyt(p) > 1, then there is a finite-memory winning strategy t* 
for player 1 for the objectives LimSupAvg(7r) > and LimlnfAvg(7r) > 0. 

2. If for all words p G E* we have Lj^(p) < 0, then there is no winning strategy for player 1 for the 
objectives LimSupAvg(7r) > and LimlnfAvg(7r) > 0. 

3. There exists a winning strategy (resp. a finite-memory winning strategy) for player 1 for the 
objectives LimSupAvg(7r) > and LimlnfAvg(7r) > iff there is a word p G E* such that L^{p) > 1. 

Proof. Note that the third item is a consequence of the first two items. The first item follows from 
Lemma 17. We now use Lemma 18 to prove the second item. Given the condition of the second item, 
let us consider the strategy r| for player 2 as described in Lemma 18. Let tt be a play consistent with 
r| . We consider two cases to complete the proof. 

— If 7T does not have infinite number of iterations, then from some point on only states q$ or qs are 
visited, and they both have only negative weights. Hence all the weights occur in it from some 
point on are non-positive and hence LimlnfAvg(7r) < LimSupAvg(7r) < 0. 

— Otherwise, tt has infinite number of iterations. Given it follows from Lemma 18 that for all 
iterations, in every round of an iteration, the sum of the weights from the beginning of the iteration 
to the current round is non-positive. Hence LimlnfAvg(7r) < LimSupAvg(7r) < 0. 

The desired result follows. □ 

Undecidability for related decision problems. It follows from Lemma 19 that the existence 
of winning strategies (resp. finite-memory winning strategies) for mean-payoff objectives with strict 
inequality is undecidable for WPGs. For general strategies the result also follows for non-strict in- 
equality by duality. We now show the undecidability for finite-memory strategy for the non-strict 
inequality as well as undecidability for stack boundedness. This is done by showing a reduction from 
the non-universality problem for WFA with threshold v = 0. The reduction is identical to the original 
reduction presented in this section. If there exists a word p G E* such that L^(p) > 0, then play- 
ing $' p l +1 rev(p) in every iteration is a finite-memory winning strategy for player-1 (also for the stack 
boundedness condition). Otherwise, for every p G E* we have L_^(p) < —1. In this case, against every 
player-1 finite-memory strategy, with memory size M, player-2 has a strategy that ensures that the 
mean-payoff is at most — Similarly, against every player-1 strategy that ensures stack height at 
most M, player-2 has a strategy that ensures that the mean-payoff at most — ^y. 

Theorem 6 Given a WPG Q, the following questions are undecidable: (1) Whether there ex- 
ists a winning strategy (resp. finite-memory winning strategy) to ensure <f N 0, where <P G 
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Fig. 4. Example WPG Q. 



{LimSupAvg, LimlnfAvg} and M£ {>,>}; and (2) whether there exists a winning strategy (resp. 
finite-memory winning strategy) to ensure <P [X along with stack boundedness, where <P 6 
{LimSupAvg, LimlnfAvg} and txiG {>,>}. 

Distinguishing facts. We now show some interesting facts about WPGs with mean-payoff objectives 
that distinguish from finite game graphs with mean-payoff objectives. 

1. Fact 1. It follows from the proof of Theorem 2 (the example of ultimately periodic words are 
not sufficient to witness values) that in general, positional (or memoryless) strategies are not 
sufficient, and infinite-memory strategies are required in general (in contrast, in finite game graphs, 
memoryless winning strategies are guaranteed to exist). 

2. Fact 2. The objectives LimSupAvg and LimlnfAvg do not coincide in general for WPGs. We show 
this in Example 1. 

3. Fact 3. We also note that pushdown mean-payoff games are very different as compared to parity 
games. For finite-state games both parity and mean-payoff conditions have the same complexity 
(both lie in NP n coNP and also in UP (~l coUP [24]), whereas for pushdown games the mean- 
payoff problem is undecidable, whereas the parity problem is EXPTIME-complete [33]. Moreover, 
for countably infinite games with finitely many priorities, for parity objectives memoryless winning 
strategies exist [31], whereas as we show (Fact 1) for mean-payoff pushdown games infinite-memory 
strategies are required. 



Example 1 We show that there exists a WPG such that player- 1 can ensure that LimlnfAvg(7r) > 2 
and player- 2 can ensure that LimlnfAvg(7r) < —2. The WPG is described as follows: Let Qi = {q},qj} 
and Qi = {qjj, qji}, and let E, the set of transitions, be as follows 

— (qj , _L, qj , puship/)) with weight — 2; 

— (qj, 7, qj, push("f)) with weight — 2; 

— (qj , 7 , qj , skip) with weight 0; 

— (qj, 7, qj, pop) with weight +4; 

— (qj, -L,q}j, skip) with weight 0; 

— (q]j, -L,q] I ,push('-y)) with weight +2; 

— (q] I , r ),q\ I ,push( r y)) with weight +2; 

— (q}j,j,qjj, skip) with weight 0; 

— (qjj,j,qjj,pop) with weight — 4; 

— (qj T , -L,qj, skip) with weight 0. 

The WPG is shown in Figure 4- H * s straight forward to verify that player- 1 can ensure 
LimSupAvg(7r) > 2, and player-2 can assure LimlnfAvg(7r) < —2. □ 
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4 Recursive Games and Modular Strategies 

In this section we will consider modular strategies in pushdown games, and modular strategies are 
more intuitive in the equivalent model of recursive game graphs. We first present the definition of 
recursive game graphs from [5] . 

Weighted recursive game graphs (WRGs). A recursive game graph A consists of a tuple 
(Ai, . . . , An) of game modules, where each game module Ai = (N, B i7 , V 2 , En i7 Exi, Si) consists of 
the following components: 

— A finite nonempty set of nodes N. 

— A nonempty set of entry nodes Eni C and a nonempty set of exit nodes Exi C Ni. 

— A set of boxes Bi. 

— Two disjoint sets and V 2 that partition the set of nodes and boxes into two sets, i.e., V^UV 2 = 
Ni U Bi and V* d V 2 = 0. The set (resp. V 2 ) denotes the places where it is the turn of player 1 
(resp. player 2) to play (i.e., choose transitions). We denote the union of and V 2 by Vi. 

— A labeling Yj : Bi — > {1, . . . , n} that assigns to every box an index of the game modules A\ . . . A n . 

— Let Calls, = {(b,e) | b G £?j,e G Enj,j = Yi(b)} denote the set of calls of module Ai and 
let RetnSj = {(b,x) | b G Bi,x G Exj,j = Yi(b)} denote the set of returns in Ai. Then, Si C 
(Ni U Retnsi) x (Ni U Calls,) is the transition relation for module Ai. 

A weighted recursive game graph (for short WRG) is a recursive game graph, equipped with a weight 
function w on the transitions. We also refer the readers to [5] for detailed description and illustration 
with figures of recursive game graphs. WLOG we shall assume that the boxes and nodes of all modules 
are disjoint. Let B = [J i Bi denote the set of all boxes, TV = [j i N denote the set of all nodes, 
En = \J i Eni denote the set of all entry nodes, Ex — \J i Exi denote the set of all exit nodes, V 1 = \J i 
(resp. V 2 = \J i V 2 ) denote the set of all places under player l's control (resp. player 2's control), and 
V = V 1 U V 2 denote the set of all vertices. We will also consider the special case of one-player WRGs, 
where either V 2 is empty (player- 1 WRGs) or V 1 is empty (player-2 WRGs). 

Configurations, paths and local history. A configuration c consists of a sequence (b\, . . . , b r , u), 
where b\, . . . , b r G B and u <E N . Intuitively, b\, . . . , b r denote the current stack (of modules), and n 
is the current node. A sequence of configurations is valid if it does not violate the transition relation. 
The configuration stack height of c is r. Let us denote by C the set of all configurations, and let Ci 
(resp. C2) denote the set of all configurations under player l's control (resp. player 2's control). A 
path 7r = (a, C2, C3, . . .) is a valid sequence of configurations. Let p = (c\,C2, ■ ■ ■ , Cfe) be a valid finite 
sequence of configurations, such that c, = (b\, . . . , b l d . , Ui), and the stack height of is dj. Let Cj 
be the first configuration with stack height di — dk, such that for every i < j < k, if Cj has stack 
height di, then Uj £ Ex (uj is not an exit node). The local history of p, denoted by Local History (p), 
is the sequence (uj lt . . . , Uj m ) such that Cj 1 — c i: Cj m = Ck, ji < ji < • ■ • < j m , and the stack height 
of Cj 1 , . . . , Cj m is exactly di. Intuitively, the local history is the sequence of nodes in a module. Note 
that by definition, for every p G C*, there exists i G {1, . . .,n} such that all the nodes that occur 
in Local History(p) belongs to V l . We say that Local History(p) G A t if all the nodes in LocalHistory(p) 
belongs to Vi. 

Global game graph and isomorphism to pushdown game graphs. The global game graph 
corresponding to a WRG A = (A\, . . . ,A n ) is the graph of all valid configurations, with an edge 
(01,02) between configurations c\ and C2 if there exists a transition from c\ to c 2 . It follows from 
the results of [5] that every recursive game graph has an isomorphic pushdown game graph that is 
computable in polynomial time. 

Plays, strategies and modular strategies. A play is played in the usual sense over the global game 
graph (which is possibly an infinite graph) . A (finite) play is a (finite) valid sequence of configurations 
(ci, c 2 , C3, . . .) (i.e., a path in the global game graph). A strategy for player 1 is a function r : C* xCi — > 
C respecting the edge relationship of the global game graph, i.e., for all i»eC and ci G Ci we have 
that (c\,t(w ■ ci)) is an edge in the global game graph. A modular strategy t for player 1 is a set 
of functions {rj}™ =1 , one for each module, where for every i, we have n : (Ni U Retris^)* — > Si. 
The function t is defined as follows: For every play prefix p we have r(p) — -^(LocalHistory^)), where 
Local History (p) G Ai. The function is the local strategy of module Ai. Intuitively, a modular strategy 
only depends on the local history, and not on the context of invocation of the module. A modular 
strategy r = {n}2 =1 is a finite-memory modular strategy if n is a finite-memory strategy for every 
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i e {1, . . . , n}. A memoryless modular strategy is denned in similar way, where every component local 
strategy is memoryless. 

Mean-payoff objectives and winning modular strategies. The weight of a finite path tt, denoted 
by w(tt) is the sum of all weights along the path. For an infinite path ir (as in the previous sections) 
we denote LimlnfAvg(7r) = liminf n _^. co 

w{-n[i,n\) (- reS p^ |_imSupAvg(7r) = limsup^gp "'(^'"D ^ where 
7r[l,n] is the initial prefix of length n. The modular winning strategy problem asks if player 1 has a 
modular strategy r such that for every play p consistent with r we have LimlnfAvg(p) > (note that 
the counter strategy of player 2 is a general strategy), and similarly for other mean-payoff objectives. 

Basic properties. We now present some basic properties of recursive game graphs. 

Non- decreasing cycles and proper cycles. A non- decreasing cycle in a recursive game graph A = 
(Ai, . . . , A n ) is a path segment from a module Ai and vertex Vi G Ai to the same module and the 
same vertex (possibly at different stack level) , such that the first occurrence of module Ai in the path 
segment does not return (i.e., does not reach an exit node) during the path segment. A non-decreasing 
cycle C is a proper cycle if the stack heights at the beginning and the end of the path segment are 
the same. 

Lemma 20 Consider a one-player WRG A = {A\, . . . , A n ) (i.e., consists of only one-player). The 
following assertions hold: 

— The WRG A has a path ir with LimlnfAvg(7r) > (resp. LimSupAvg(7r) > 0) iff there exists a 
non- decreasing cycle with positive weight. 

— The WRG A has a path tt with LimlnfAvg(7r) < (resp. LimSupAvg(7r) < 0) iff there exists a 
non- decreasing cycle with negative weight. 

Proof. The first item follows from (i) the isomorphism of one-player WRGs and weighted push- 
down systems (WPSs), (ii) the correspondence positive non-decreasing cycles and good cycles for 
WPSs, and (iii) the results established in Section 2 showing equivalence of existence of a path tt with 
LimlnfAvg(7r) > (resp. LimSupAvg(7r) > 0) and existence of good cycles in a WPS. The second item 
follows from the duality of LimlnfAvg(7r) > and LimSupAvg(7r) < 0. □ 

WRG given finite-memory strategies. Given a WRG A, let r = {ri}" =1 be a finite- memory modular 
strategy. Let M, be the set of memory states of strategy Tj, i.e., tj is described as a deterministic 
transducer with state space Mj. The one-player WRG (player-2 WRG) given r is the tuple A T = 
(A] 1 = A\ x Mi, . . . , A 7 ^ = A n x M n ), where each A^ = Ai x M, is obtained as the synchronous 
product of Ai and the deterministic transducer describing the local strategy n. The weights of the 
transitions are specified according to the weight function of A. Note that if r is a memoryless modular 
strategy, then A T is obtained as a sub-gamegraph of A. 

Lemma 21 Given a WRG A and a modular strategy t, every (finite or infinite) path in the one-player 
WRG A T is a (finite or infinite) play in A consistent with t, and vice versa. 

4.1 Decidability of the modular winning strategy problem 

In this section we will establish the decidability of the existence of modular winning strategy problem. 
In the following section we will establish the NP upper bound, and finally show NP-hardness. We 
start with objective LimlnfAvg > 0, and then show the result for objective LimSupAvg > 0. The result 
for mean-payoff objectives with strict inequality will also easily follow from our results. 

Objective LimlnfAvg > 0. For the decidability result, we will show the existence of cycle independent 
modular winning strategies, and the result will also be useful to establish the complexity results. We 
start with the notion of a cycle free path in a graph. 

Cycle free path. Let G — (V, E) be a simple (no parallel edges) directed graph. Wc define the 
operator CycleFree : V* — >• V* in the following way: let n = (v\, V2, ■ ■ ■ , v n ) be a finite path in G. 

— CycleFree(7r) = n if tt is a simple path (i.e., with no cycles). 

— Otherwise we define CycleFree inductively as follows. Let CycleFree(«i . . . v n -\) = U\U2 ■ ■ -u m . 
Let i be the first index such that v n = Mj. If such an index does not exist, then CycleFree(7r) = 
u\U2 ■ ■ ■ u m v n . Otherwise CycleFree(7r) = uiu 2 ...Ui. Intuitively, the CycleFree operator takes a 
finite path and returns a simple path by removing simple cycles according to order of appearance. 
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Cycle independent modular strategy. Given a recursive game graph, a local strategy Ti for module 
Ai is a cycle independent local strategy, if for every p e V* we have n(p) = Ti(CycleFree(p)). A modular 
strategy r = {ri}™ =1 is a cycle independent modular strategy if n is a cycle independent local strategy 
for every i e {1, . . . , n}. 

Observation 1 For a recursive game graph A = (A\, . . . ,A n ), there exists at most \V^ V<{2 different 
cycle independent modular strategies, where \V\ is the number of vertices in A. 

The main result of this section is that if there is a modular winning strategy, then there is a cycle 
independent modular winning strategy. To establish the result we introduce the notion of manipulated 
paths, using rewind, fast forward and simulation operations. 

Manipulated paths, rewind, fast forward and simulation operations. Let r = {ri}™ =1 be a 
modular winning strategy for the objective LimlnfAvg > 0, and let e > be an arbitrary constant. Let 
7r m = 7r m _i • ni be a play prefix at round to, that ends at node rij G Ai . The manipulated play prefix 
of 7r m according to r and e, denoted by Man^ (7r m ), is defined inductively as follows: Let Man^(7r m _i) 
be the manipulated play prefix at round m — 1. Then ManJ (7r m ) is obtained from Man^ (7T m _i) and 
Hi by one of the following operations. 

1. Rewind operation: The condition for the rewind operation is CycleFree(Man^(7r m _i)) • closes a 
proper cycle in the top module Ai. If the rewind condition holds, then Man^(7r m ) is formed from 
Man^(7r m _i) • ni by removing the proper cycle suffix from Man^(7r m _i) • n^. Intuitively the rewind 
operation rewinds the path by chopping off the cycle in the end. 

2. Fast forward operation: Let h = Man^(7r m _i) • n^. The fast forward condition for history h that 
ends at node ni is as follows: there exists a play prefix h-ir'(h) consistent with r such that n, -it' '(h) 
is a proper cycle with average weight less than — e. In order to be precise, we define 7r'(/i) as the 
first such prefix according to lexicographic ordering of the prefixes. If the rewind condition does 
not hold, and the fast forward condition holds for ho, then construct hi = ho ■ it' (ho). Continue 
the process and build hi = hi-\ ■ 7r'(/ii_i), as long as satisfies the fast forward condition. If 
there exists a minimal index igN such that hi does not satisfy the fast forward condition, then 
we define Man^(7r m ) = hi. Otherwise, Man^ (7r m ) is undefined (not well defined), and we say that 
the process is stuck in the fast forward operation. 

3. Simulation operation: Else, if the rewind and fast forward conditions do not hold, then we have 
Man[(7r m ) = Man[(7r m _i) • n,. 

In the following lemma we establish consistency and well-definedness of the manipulated operation 
for a winning strategy. 

Lemma 22 Let t be a winning strategy (a general winning strategy, not necessarily modular) for the 
objective LimlnfAvg > 0. Let e > be an arbitrary constant. We define a strategy a in the following 
way: for a history it' we have o-(n r ) = T(Man^(7r')). Let ir m be a play prefix of length to that is 
consistent with a. Then the following assertions hold: 

— Marig (7r m ) is well defined, i.e., the process does not get stuck in the fast forward operation. 

— Ma rig (7r m ) is consistent with r. 

Proof. We shall prove both the items by induction on m. In the base case when m = (i.e., empty 
play prefix), all the claims are trivially satisfied. We now consider the inductive case with to > 0. Let 
7Tm = Tm-i ■ rii, for some ni e Ai, be a play prefix consistent with a. By the inductive hypothesis 
ManJ" (7r m _i) is well defined and consistent with r. Then Man^ (7r m ) is computed by performing one of 
the following operations 

— Rewind operation: In this case clearly Man^(7T m ) is well defined. In addition Man^ (7r m ) is a prefix 
of Marig (7r m _i), which is by the inductive hypothesis consistent with t, hence also Man^(7r m ) is 
consistent with r. 

— Fast forward operation: Towards contradiction, let us assume that the fast forward process enters 
infinite loop. We consider the prefix h = h ■ ni, where ho = Man^ (7T m _i), and let vo be the last 
vertex in ho- The prefix ho is consistent with r for the following reason: Man^ (7r m _i) is consistent 
with t by the inductive hypothesis, and if Vo is under player l's control, then r(ManJ (w m -i)) = rii 
(as 7r m consistent with a), and otherwise tJq is under player 2's control, and since there is a 
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transition from Vo to rn (as 7r m is a play prefix) the consistency of ho follows. The prefix ho has 
an infinite sequence of extensions ir 1 , tt 2 , . . . such that the infinite play h = ho^ir 2 . . . 7r 5 7r' +1 . . . 
is consistent with t and Avg(7r- J ) < — e for every j G N (by the infinite loop of the fast forward 
operation). Hence, by definition, LimlnfAvg(/i) < — e < 0. 4 Thus we get that there exists a play 
consistent with r that is not winning for the objective LimlnfAvg > 0, which contradicts the 
assumption that r is a winning strategy. Hence, the fast forward process always terminates. It 
follows that Man^(7r m ) is well defined and also by definition of the fast forward operation it is 
consistent with r. 

— Simulation operation: By definition, Man^(7r m ) = Man^(7r m _i) • n^. Let ho = Man^(7r m _i), and 
let vo be the last vertex in ho- The prefix Man^ (7r m ) is consistent with r for the following reason: 
ManJ"(7r m _i) is consistent with r by the inductive hypothesis, and if vo is under player l's control, 
then r(Marig (7r m _i)) = n^, and otherwise vo is under player 2's control, and there is a transition 
from v to n,. Thus we have the consistency of Man^ (7r m ), and the well-definedness is trivial. 

Hence we have that ManJ(7r m ) is both consistent with r and well defined. □ 

In the following lemma we obtain a bound of the average of the play prefixes obtained from the 
manipulated operation of a winning strategy. 

Lemma 23 Let r be a winning strategy (a general winning strategy, not necessarily modular) for the 
objective LimlnfAvg > 0. Let e > be an arbitrary constant. We define a strategy a in the following 
way: for a history tt' we have o-(tt') = T(Man^(7r')). Let n m be a play prefix of length to that is 
consistent with a. Then we have w(ir m ) > w(ManJ(7r m )) — e • |7r m |. 

Proof. The following claim is the key for the proof. 

Claim. Every time a rewind operation is done, the cycle C, for which ManJ(7r m ) C = Man^(7r m _i) -m, 
satisfies that Avg(C) > — e. 

We first prove the claim. Towards contradiction, assume that Avg(C) < — e. Let j < to be the 
first index for which ManJ(7rj) = Man^ (7r m ). Note that such index must exist. We first argue that 
ManJ(7r m ) • C = ManJ(7r m _i) • in is consistent with r: (i) Man^(7r m _i) is consistent with r (by 
Lemma 22) and (ii) let ho = ManJ(7r m _i), and let vo be the last vertex in h : if Uo is under player l's 
control, then T(ManJ (ftm-i)) = n^, and otherwise vo is under player 2's control, and there is a 
transition from tJ to n,. Thus we have the consistency of ManJ(7T TO ) • C — Man^(7T TO _i) • rij, and 
it follows that Man^(7Tj) • C is also consistent with r. Hence, since Avg(C) < — e, a fast forward 
operation had occurred in round j (note that it is not possible that Man^ (iTj) was obtained after a 
rewind operation, since j is the first index for which ManJ(7rj) = Man^(7r m )). Hence it is not possible 
that Marig (7Tj) = Man^ (7r m ), since at the very least, Man^ (7r m ) • C is a prefix of ManJ(7Tj). Thus, for 
every j < to we have Man^(7Tj) ^ ManJ(7r m ), and the contradiction is obtained. 

We now complete the proof of the lemma using the claim. We note that difference between 7r m 
and Ma rig (7r m ) contains only (i) cycles with negative weight that were added to ManJ(7r m ) (by fast 
forward operation) or (ii) cycles with average weight at most — e and length at most |7r m | that were 
chopped from Man^ (7r m ) (by rewind operation). The desired result follows. □ 

We now show that from a winning strategy for the objective LimlnfAvg > 0, the strategy obtained 
using manipulated operation is winning for the objective LimSupAvg > —2 • e. 

Lemma 24 Let r be a winning strategy (a general winning strategy, not necessarily modular) for the 
objective LimlnfAvg > 0. Let e > be an arbitrary constant. We define a strategy a in the following 
way: for a history n' we have <t(tt') = r(Man^(7r')). Then a is a winning strategy for the objective 
LimSupAvg > —2 • e. 

Proof. Let tt be a play consistent with cr, and let 7r m be the play prefix until round to. We consider 
two cases to complete the proof. 

1. In the first case, there exists a constant no £ N such that for infinitely many indices mi, m,2, ■ ■ ■ , 
we have |ManJ(7r mi )| < no- In this case, due to Lemma 23, in rounds mi,m,2, ■ ■ ■ we get that 
w{^ mi ) > —no - W — e ■ \TT mi \- Hence, by definition, LimSupAvg(7r) > — e > — 2 • e. 

4 only this inequality need not hold for LimSupAvg(ft) 
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2. In the second case, for every i > there exists £i G N, such that for every m > I, we have 
ManJ(7r m )| > i. By the definition of the manipulation operations, we get that ManJ(7r^)[0,z] = 
Man^(7T£ i+ i)[0, i], i.e., the prefix upto length i coincides. Denote pi = ManJ(7r^)[i] the i-th position 
of ManJ(7r^). Due to Lemma 22 the infinite play p = p\p2 ... is consistent with r. Since r is a win- 
ning strategy we get that LimlnfAvg^) > 0. Hence there exists infinitely many indices mi, mi, . . . 
for which Avg(ManJ(7r mi )) > — e, and therefore, due to Lemma 23, we get that Avg(7r m J > —2 • e. 
Hence by definition of LimSupAvg, we obtain LimSupAvg(7r) > — 2 • e. 

This concludes the proof of the lemma. □ 

Lemma 25 Given a WRG A, let r be a modular winning strategy for the objective LimlnfAvg > 0. 
Let e > be an arbitrary constant. We define a strategy a in the following way: for a history ir' we 
have <r(ir') = r(ManJ(7r')). Then a is a cycle independent modular strategy. 

Proof. In order to verify that a is a modular strategy, we observe that LocalHistory(Man^(7r)) is 
computable (maybe not effectively) from Local History (ir) and that r is a modular strategy. 

To verify that a is a cycle independent strategy, we observe that if tt and tt ■ tt c are consistent with 
a and 7r c is a cycle in CycleFree(7r) • 7r c , then Man^(7r • 7r c ) = Man^ (tt) as tt c will be chopped by the 
rewind operation. □ 

Lemma 26 Given a WRG A, if there exists a modular winning strategy for the objective LimlnfAvg > 
for player 1, then there exists a cycle independent modular winning strategy for the objective for 
player 1. 

Proof. Let r be a modular winning strategy for the objective LimlnfAvg > 0. For every e > 0, define 
the strategy a e in the following way: for a history tt' we have o- c (tt') = r(ManJ(7r')). By Lemma 24 
and Lemma 25, for every e > we have a e is a cycle independent modular winning strategy for the 
objective LimSupAvg > — 2 • e. Since there are only a bounded number of cycle independent modular 
strategies (by Observation 1), it must be the case that one of the modular strategies is a winning 
strategy for the LimSupAvg > objective. Let a be that strategy. Let A a be the player-2 WRG 
obtained given the strategy a. As o is a winning strategy for the objective LimSupAvg > 0, then due 
to Lemma 21 and Lemma 20, the graph A a does not have a negative non-decreasing cycle. Hence due 
to Lemma 21 and Lemma 20, the strategy a is winning also for the objective LimlnfAvg > 0. This 
completes the proof of the result. □ 

The next theorem is an immediate consequence of Lemma 26. 

Theorem 7 Given a WRG A, the problem of deciding if player 1 has a modular winning strategy for 
the objective LimlnfAvg > is decidable. 

Proof. By Lemma 26 it is enough to check if player 1 has a cycle independent modular strategy. As 
the number of such strategies is bounded (by Observation 1), it is enough to construct the graph 
A T for every cycle independent modular strategy r and check if in A T there exists a path tt with 
LimlnfAvg(7r) < (this check is achieved using the algorithms of Section 2). □ 

Objective LimSupAvg > 0. The proof for the objective LimSupAvg > will reuse many parts of 
the proof for the objective LimlnfAvg > 0, however, some parts of the proof are different and wc 
present them below. In fact we will show for modular winning strategies the objective LimSupAvg > 
coincides with the objective LimlnfAvg > 0. For the proof we need the notion of non- negative cycle 
free local history, which we define below. 

Non-negative cycle free local history operator. Consider a WRG A and let r be a modular 
strategy, and tt be a path in A, consistent with r, that begins at the entry of module Ai and ends at 
module Ai (in the same stack height). The non-negative cycle free local history operator is defined as 
follows: 

1. For |LocalHistory(7r)| =lwe have NonNegCFLocalHistory T (7r) = Local History (tt). 

2. For |LocalHistory(7r)| > 1, let tt = ir Vi such that NonNegCFLocalHistory T (7r ) = u ui . . . u m . Let 
j G {0, . . . , m} be the first index such that Uj = Vi, and every sub-play 7r* consistent with r with 
local history ujUj + i . . . u m Vi is a cycle with non-negative total weight. If such index j exists, then 
NonNegCFLocalHistory T (7r) = u . . . Uj, otherwise NonNegCFLocalHistory T (7r) = m ui . . . u m Vi. 
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Informally, the NonNegCFLocalHistory T operator removes cycles that are assured to have non-negative 
weight from the local history. 

Non-negative cycle independent modular strategy. Given a modular strategy r, the non- 
negative cycle independent modular strategy a of t, is defined as follows: For a local history p we 
have a(p) = r(NonNegCFLocalHistory T (/?)). We now present some notations required for the proofs. 
Sure non-negative cycle and proper simple cycle. Given a modular strategy r, a path tt — v vi . . . v m 
is a sure non-negative cycle if tt is a proper cycle consistent with r, and every path tt' consistent with 
t such that Local History (tt) = LocalHistory(7r') is a proper cycle with non-negative weight. A path tt 
is a proper simple cycle if tt is a proper cycle, and Local History (tt) is a simple cycle. 

Lemma 27 If t is a modular winning strategy for the objective LimSupAvg > 0, then the non-negative 
cycle independent modular strategy a of r is also a winning strategy for the objective LimSupAvg > 0. 

Proof. The proof is essentially similar to the proof of the result for the objective LimlnfAvg > 0, 
and we present a succinct argument (as it is very similar to the previous proofs for LimlnfAvg > 0). 
As previously, we define manipulated operations on history such that every sure non-negative cycle is 
chopped (by the rewind operation) from the history. By definition, the non-negative cycle independent 
strategy a of r makes choices according to the choices of r on the manipulated history. The weight of 
the manipulated history, in every round, is at most the weight of the original history, since only non- 
negative cycles are chopped. By similar arguments to those presented in Lemma 24, we get that the 
LimSupAvg of the original history is at most 0, and thus the non-negative cycle independent strategy 
g of r is a winning strategy. □ 

In the following lemmas we establish that for modular strategies the objectives LimSupAvg > 
and LimlnfAvg > coincide. 

Lemma 28 Given a WRG A, let a be a non-negative cycle independent modular strategy. Let Ai be 
a module in A. Then there exists n a £ N and So- > 0, such that for every possible non-negative cycle 
free local histories ho and hi of the module Ai, and for every sub-play p, consistent with a such that 

— p is a proper cycle with negative weight; and 

— the non-negative cycle free local history of Ai before (resp. after) p was played is ho (resp. hi); 

there exists a sub-play Ph M , consistent with a and satisfies the items above, such that every play-prefix 
of Ph M w ith length at least n a has an average weight of at most —5 a . 

Proof. Let (&i,ni), . . . , (b m , n m ) be the pairs of all boxes and their return nodes that appear in module 
Ai. For every pair (bj, rij), let TTb jtnj be the shortest play, consistent with a, with minimal weight from 
bj to rij. If such path exists we denote W(bj,nj) = w(iri, jtnj ). Let X be the maximal such weight. W.l.o.g 
all the weights of the edges occur in Ai are at most X, and X > 0. 

For every bj, rij such that there exists a play from bj to nj consistent with a, but shortest minimal 
play does not exist, we denote by 7r(, J)Tlj the shortest play consistent with a that leads from bj to nj 
with weight at most — 20 • \ Vi \ ■ X, (where \Vi\ is the size of the vertex set in Ai). Let LJ be the longest 
path among all 71"^ • Let p be a sub-play consistent with a such that p is a proper cycle in module 
Ai with negative weight. Let h (resp. hi) be the non-negative cycle free history of Ai before (resp. 
after) playing p. 

First, we form p' from p by removing all the sure non-negative cycles from p. Clearly, (i) p' has 
negative weight; and (ii) p' is consistent with a, because p is consistent with a and a is a non-negative 
cycle independent strategy. Moreover, the non-negative cycle free local history after playing p' is the 
same as after playing p. Next we form p" from p' by replacing every sub-play from bj to rij (such that 
nj is the first time the sub-play enters Ai) in p' with TTb^ny Again, p" is consistent with a, since o is 
a modular strategy. In addition, the local history of Ai was not changed at all. 

We claim that p" does not contain simple proper non- negative cycles in module Ai. Indeed, towards 
contradiction let V1V2 ■ ■ ■ v m be the local history of the first such cycle (note that m < \Vi\). Note that 
if the cycle contains a sub-play 7Tfy )TlJ such that W(b^ nj ) = —20 • \Vi\ ■ A, then the cycle cannot be 
with non- negative weight. Hence it follows that this cycle is the cycle with minimal weight among all 
simple cycles with local history V1V2 ■ ■ -v m . Hence this cycle is sure non- negative, which contradicts 
the fact that viv 2 ■ ■ .v m is a local history of sub-play of p" . Thus every simple proper cycle in p" has 
a negative weight. Moreover, the length of every simple proper cycle in p" is at most \Vi \ ■ \n\. Hence 
every sub-play of p" with length at least n a = (\Vi\ ■ \LT\ ■ X) 2 will have an average weight of at most 
Sfj = — TyTTjfix - Note that n a and 5 a do not depend on p, hence the desired result follows. □ 
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Lemma 29 Let a be a non-negative cycle independent modular strategy. If there exists a play p 
consistent with a such that the suffix of p is an infinite sequence of proper cycles C\,Ci,--- with 
negative weights, then a is not a winning strategy for the objective LimSupAvg > 0. 

Proof. Let p = PqC\Ci . . . Cj . . . , and let Ai and rn be the module and the vertex, respectively, that all 
the cycles begin and end in. Let n a , 8 a be the constants from Lemma 28. Let /iq be the non- negative 
cycle free local history of Ai before cycle Cj is played, and h\ be the non-negative cycle free local 
history of A, after cycle Cj is played. By Lemma 28, for every cycle d there exists a cycle C h i ^ 
consistent with a, with negative weight, and the average weight of every sub-play of C^. h i longer 
than n a is at most — S a . The play p' = p^C\C2 ■ ■ ■ Ci-\C h i h \ Cj+i ... is consistent with a, since G^; ^ 
is consistent with the initial non- negative cycle free local history h % Q , and the non- negative cycle free 
local history after playing C h % ^« is h\ . Since a is a non- negative cycle independent strategy, the play 
p* = poC h o h oC h i jh i . . . Cfri^i ... is consistent with a. On the other hand, it is straight forward to 
verify that we have LimSupAvg(p*) < max{ — — , — S a } < 0. Hence a is not a winning strategy for the 
objective LimSupAvg > 0, and the desired result follows. □ 

Lemma 30 If player 1 has a modular winning strategy for the objective LimSupAvg > 7 then for every 
e > 0, player 1 has a cycle independent modular winning strategy a for the objective LimSupAvg > — e. 

Proof. Let r* be a modular winning strategy for the objective LimSupAvg > 0, and let r be the 
non- negative cycle independent strategy of r*. For e > 0, consider the cycle independent modular 
strategy a such that for history 7r we have a(w) = ManJ(7r). By Lemma 28 the strategy r is also a 
winning strategy. We note that all the arguments for the objective LimlnfAvg > also hold for the 
objective LimSupAvg > 0, other than the inequality (mentioned as footnote) in Lemma 22. We replace 
the inequality (mentioned as footnote) of Lemma 22 with Lemma 29, and repeat exactly the same 
arguments for the objective LimlnfAvg > (up to Lemma 26) to obtain the desired result. □ 

We obtain the following result as a corollary, and all the desired results follow for the objective 
LimSupAvg > 0. 

Corollary 1 Given a WRG A, there exists a modular winning strategy for the objective LimSupAvg > 
iff there exists a modular winning strategy for the objective LimlnfAvg > 0. 

4.2 Modular winning strategy problem in NP 

In this section we will show that the modular winning strategy problem is in NP. To show the result 
we introduce the notion of a signature game. 

The signature game. Let G = ((V, E), (V\, V2)) be a finite two-player game graph (on finite directed 
graph (V, E)) with vertex set V, edge set E, and partition (Vi,V2) of the vertex set into player-1 
(resp. player-2) vertex set V\ (resp. V2). Let the game graph be equipped a weight function w : E — >• 
Z U {— ui}. Let the initial vertex be wo £ V. Let v — {y\, . . . , v\y\) be a threshold vector such that 
all i/jEZU {+00, — uj}. The weight of a finite path in G is the sum of the edge weights of the path, 
according to the following convention: — lo + z = —uj, for any z e ZU{— uj}. A signature game consists 
of a tuple (G, f), where G is a two-player game graph, and v is the threshold vector. For a play 
P = P0P1P2 ■ ■ ■ Pjpj+i ■ • ■ , player 1 is the winner if both the following two conditions hold: 

— The play p does not contain a negative cycle, or a cycle that has an edge with weight — ui. 

— For every j 6 N, if pj = Vi (i.e., the j-index of the play is the i-th vertex), then w(poPiP2 ■ ■ ■ Pj) > 
fi, according to the convention (i) —ui < z for every z £ Z, and (ii) z, —ui < +00 for every z 6 Z. 
In other words, the sum of the weights upto any index j (with i-th vertex in index j) must be at 
least Ui, and no — w-edge must be visited unless Ui = —uj. 

We will consider signature games such that if Ui e Z, then Ui > — 2 • W ■ \V\, where W is the maximum 
absolute values of the integer weights in graph G. If for a vertex v, the threshold value is +00, then 
to ensure winning player 1 must ensure that v is never visited. In other words, vertices with +00 
threshold must be avoided (it can be interpreted as a safety objective with +00 vertices as non-safe 
vertices to be avoided) . Our first goal is to show that mcmoryless winning strategies exist in signature 
games, and the result will be obtained by a reduction to finite-state mean-payoff games. Given a 
signature game (G, u) we define an auxiliary finite-state mean-payoff game as follows. 
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From signature game (G, v) to auxiliary mean-payoff game G v . Given a signature game 
(G, v) we construct a finite-state auxiliary mean-payoff game G v = ((V, E), (Fi, V^)), with an weight 
function w as follows: let the signature game graph be G = ((V, E), (Vi, V2)) and the weight function 
in G be wq- Then we have the following components in the auxiliary game: 

— (Vertex set and partition). V = V x {1, 2}; and V\ = V\ x {1}. 

— (Edges). E = {((«, 1), (v, 2)) | e £} U {((v, 2), («, 1)) | v e V} U {((«,, 2), (v , 1)) | 1* £ 
V,i/< ^ -w}. 

— (Weight function). If wg(u,v) =/= —lj, then wJ((u, 1), (u, 2)) = wq(u,v); otherwise (we have 
wg(u 7 v) = — lj) we set w((u, 1), (w, 2)) = —10 • W ■ \V\. Moreover, w((vi, 2), (w , 1)) = —v% and all 
the other edges are assigned with zero weight. Note that if vi = +00, then w assigns weight —00, 
and to win player 1 must avoid such edges (can be interpreted as a safety condition). 

Informally, the auxiliary mean-payoff game is constructed from the signature game by adding for every 
vertex V{ a fresh copy (vertex (v{, 2) and the original vertex is represented as (vi, 1)), and an option 
for player 2 to return to the initial vertex (v , 1) "paying" cost —Vi (whenever i/j ^ —lj). Thus, if at 
any round, the play is on vertex (vi, 2), and the sum of the weights since the last visit to («o, 1) is 
less than i/j, then player 2 can ensure that a negative cycle is completed. The mean-payoff objective 
of player 1 is to ensure non- negative average payoff. Also note that player 1 must avoid the —00 edge 
weights, and equivalently it can be treated as a mean-payoff safety game. In the following lemma we 
establish the relation of the signature game and the auxiliary game. 

Lemma 31 Let (G, v) be a signature game such that Vi > — 2 • W ■ \V\ for every Vi G Z, and let G v 
be the corresponding auxiliary mean-payoff game. Then the following statements are equivalent: 

1. Player 1 is the winner in the auxiliary mean-payoff game G v (i.e., player I can ensure non-negative 
mean-payoff). 

2. Player 1 has a memoryless winning strategy in the signature game (G, v). 

3. Player 1 is the winner in the signature game (G,v). 

Proof. We first prove that item 1 implies item 2. 

1. In order to prove that item 1 implies item 2, let us assume that player 1 is the winner in the 
auxiliary mean-payoff game. Note that the auxiliary mean-payoff game is equivalently a mean- 
payoff safety game, and therefore player 1 has a memoryless winning strategy r in the auxiliary 
mean-payoff game [16] (a mean-payoff safety game is easily transformed to a mean-payoff game 
by making the non-safe vertices absorbing with negative weights). Hence the memoryless strategy 
t ensures that edges with weight — 00 are never visited. Towards contradiction, let us assume that 
t is not a winning strategy for the signature game (note that t is also a well defined player 1 
strategy in the signature game choosing edges in copy 1 according to r). Therefore one of the 
following two cases occur. 

— Case 1: There exists a finite play prefix p that is consistent with r, which starts from the initial 
«o to some vertex Vi G V with sum of weights less than Vi . In this case, either p goes through 
an — to edge, or wa(p) <v%- If p goes through an — to edge in G, then the weight of p in G v is 
at most —9 • W ■ \V\, since w.l.o.g we can assume that p does not have positive cycles (as r is 
memoryless). As — 9 • W ■ \V\ < Vi, it follows that the path (p ■ ((v i} 2), (u , l))) w is consistent 
with t and has a negative mean-payoff in the auxiliary game. This contradicts the assumption 
that t is a winning strategy. If p does not go through an — ui edge, then wg(p) < v% and again 
(p-((vi, 2), (vo, l))) w is consistent with r and has a negative mean-payoff in the auxiliary game. 
This is again a contradiction that r is a winning strategy, and concludes the proof of the first 
case. 

— Case 2: There exists a finite play prefix p = pi ■ P2 that is consistent with t, such that pi is a 
negative cycle (or a cycle with — lj edge) in the signature game graph. If p 2 does not contain 
an — lj edge e, then by definition, p^ is a negative cycle also in the auxiliary game. Otherwise, 
P2 contains an — lj edge e, and then again p 2 is a negative cycle in the auxiliary game, as 
w.l.o.g we can assume that p 2 does not contain positive cycles, and since w(e) < —10 • W ■ \ V\. 
Thus the play p\ ■ (p-i)^ is consistent with r and has a negative mean-payoff in the auxiliary 
game. This contradicts the assumption that r is a winning strategy, and completes the proof. 

2. Item 2 trivially implies item 3. 
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3. We now show that item 3 immediately implies item 1. It is straight forward to verify that if player 1 
plays according to the signature game winning strategy in every round, then a negative cycle will 
not be formed in the auxiliary game (as a negative cycle is not formed in the signature game, and 
the threshold vector is always satisfied in the signature game) and a vertex with threshold +oc 
will never be reached. Hence the mean-payoff of the play in the auxiliary mean-payoff game will 
be non- negative and — oo edges will never be visited. This shows that item 3 implies item 1. 

This completes the proof. □ 

Lemma 32 Let (G,v) be a signature game such that Vi > —2-W ■ \V\ for every Vi £ Z. There is 
a winning strategy for player 1 in the signature game (G, u) iff player 1 has a memoryless winning 
strategy. 

Proof. Follows from Lemma 31. □ 

Signature games to memoryless modular strategies. We will now use the existence of cycle 
independent modular winning strategies, and memoryless strategies in signature games to show exis- 
tence of memoryless modular strategies. For simplicity we will consider recursive game graphs where 
every module has a single entry, and a simple polynomial reduction from multi-entry recursive game 
graphs to single entry recursive game graphs is established in [5]. To prove the result of memoryless 
modular strategies we define the signature games for modular strategies. 

Signature games for modular strategies. Consider a WRG A = {A\, A2, . . . , A n ) and let r = 

{ T i}i=i be a modular strategy. Consider a module Ai in A. Let b £ E>i be a box in module A i7 which 
invokes the module A, , and let rii £ Ni be a node in module Ai that is connected to one exit of Aj , 
which is reachable according to the strategy r. We denote by wf )n . the minimal weight of all plays 
according to t that begins at the call to box b and ends at nj (in the same stack height), and do not 
visit any other vertices in Ai (in the same stack height). If such minimal weight does not exist, then 
let wj )n . = —ijj. For every module Ai, we form a finite two-player game graph Ga {1 with a weight 
function as follows: (i) in the module Ai we add an edge from every box b to every return node m 
with weight and add a self loop, with weight 0, to every exit node; and (ii) every box is now 

interpreted as a player-2 vertex. Note that the local strategy Ti is a well defined player- 1 strategy in 
the game graph Ga^ For a vertex Vj e Vi, (i) if Vj is visited along a play consistent with n, then let 
t]j denote the maximal value such that in every round of a play according to n on G At , that begins 
in the entry node of Ai, and is currently at vertex Vj £ Vi, the sum of weights from the beginning of 
the play is at least rjj; and (ii) otherwise, Vj is never visited along all plays consistent with tj, then 
r]j = +00 (note that this is like a safety condition to ensure Vj is not visited). The signature game for 
t on module Ai consists of the game graph Ga { and the threshold vector v l £ ({— u>, +00} U %)^ Vi \ 
such that h> % j — f]j for all vertices vj £ Vi. We denote by (GU i; v % , r) the signature game obtained given 
the modular strategy r on module Ai. We first establish some basic properties in the following lemma, 
and then in the following lemma we establish properties of the winning strategies in the signature 
games from modular strategies. 

Lemma 33 Let A be a WRG. If t is a cycle independent modular winning strategy for the objective 
LimlnfAvg > 0, then for every module Ai the following assertions hold: 

— Ti is a winning strategy for the signature game (G Ai,v l ,t); and 

— the integer coefficients of v l are at least —2-W ■ \Vi\. 

Proof. The first fact of the lemma follows from the facts that every path according to n has sum of 
weights at least v l and does not contain negative cycles (by the definition of signature game given 
r as t is a winning strategy), The second fact of the lemma follows from the fact that every path 
according to r, does not contain negative cycle, as n is a cycle independent modular winning strategy. 

□ 

Lemma 34 Let A be a WRG. Let t = {Ti}™ =1 be a cycle independent modular winning strategy in 
A for the objective LimlnfAvg > 0. Let a = {<7j}™ =1 be a modular strategy such that Oi is a winning 
strategy for the signature game (G Ai,v l ,t). Then for every play p a , consistent with a, which starts 
from the entry node of a module Ai to a node n£ in the same module ( and possibly goes through box 
nodes), there exists a play p T , consistent with t, from the same entry node to the same node ne, such 
that w{p a ) > w{p T ). In addition, the path p a does not contain a negative proper cycle. 
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Proof. The proof is by induction on the additional stack height of p a . 

— Base case: Additional stack height is 0. In this case the play p a have only edges from Ai, and the 
weight of the play is identical to the weight of the same play in (GU 4 , u % , t). The play p a does not 
visit a vertex with threshold +00, otherwise Ui would not be a winning strategy in the signature 
game {GauV 1 , t). Hence, by definition, there exists a play p T , consistent with t from the entry 
node to ng with weight at most v\. Since ai is a winning strategy in the signature game, and p a 
is consistent with <7j, we get that w(p a ) > v\. Therefore w(p a ) > w(p T ). Since ai is a winning 
strategy in the signature game, and the path p a is consistent with ai also in graph G 'a { , we get 
that p a docs not contain negative cycles. 

— Inductive step: Additional stack height > 0. For simplicity, we first assume that p a goes only 
through one box node in the module Ai (in the first stack level). Let node b be that box, and let 
node u G Ai be the return node in that path. Let pi u be the sub-play from the entry node of b to 
node u. Recall that w£ M is the minimal weight among all plays consistent with r between b and 
u. Let Aj be the module invoked by b, and let u' be the exit node that leads to the return node 
u in Ai. As the additional stack height from the entry node of Aj to u' is strictly smaller than 
the additional stack height of p a , it follows from the inductive hypothesis that there exists a path 
consistent with r between these two nodes with weight at most w{p1 u ). Hence wJ )U < w(p1 u ). 
Thus, the weight of p a is bounded from below by the induced path of p a over the signature game 
[Gai , v % , r) . Thus, by the definition of the signature game there exists a path p T as desired. In 
addition, by the inductive hypothesis, the path p% u does not contain proper negative cycle, and by 
the same arguments as above, there is also no negative proper cycle in module Ai. The case where 
p a goes through more then one box, is a straight forward extension of the argument presented 
above. 

Thus we have the desired result. □ 

Lemma 35 Let A be a WRG. Let r = {ri}f =1 be a cycle independent modular winning strategy in 
A for the objective LimlnfAvg > 0. Let a = {<7i}™ =1 be a memoryless modular strategy such that ai is 
a memoryless winning strategy for the signature game {GAi,v l ,T). Then a is a memoryless modular 
winning strategy in A for the objective LimlnfAvg > 0. 

Proof. Let A a be the player-2 WRG obtained by fixing the memoryless modular strategy a in A. 
Assume towards contradiction that A a has a reachable non-decreasing negative cycle C, and let p be 
a finite path that leads to first vertex of C. By Lemma 34 it follows that C cannot be a proper cycle. 

First, we argue that in A there exists a finite path p T from the first vertex of p to the last vertex 
of p that is consistent with r. Indeed, by Lemma 34, between every entry node and box node in p 
there exists a path consistent with r, and finally there also exists such a path between the last entry 
node and the last node of p. 

Second, to achieve the contradiction we will show that r is not a winning strategy. Let e\ be the 
first entry node in C (it must exist as C is not a proper cycle), and rii be the last (and first) node in 
C (note that C is not a proper cycle, and hence this node is well defined). Note that for every m G N, 
the path C m is a non-decreasing cycle that is consistent with a (as a is a memoryless strategy). Let 
p ei ,m be the path from e\ to rii. Let p nuei be the path from m to first appearance of e\ in C. We 
consider the path p* = p ei , ni ■ C m ■ p ni , ei , for m = 2 • W ■ {\p ei ,n t \ + IPn^eJ)- This is a path that is 
(i) consistent with a, (ii) begins and ends in the entry node e\ of the same module (not necessarily 
in the same stack height). Let bi,b 2 , ■ ■ - bg be the boxes occur in the path. By Lemma 34, for every 
k there exists a path pl itbi+1 consistent with r such that w(pl i b ) < w(p b . b ). Hence there exists 
a path pi that is consistent with r from e\ to e\ such that the sum of the weights is negative. As 
t is a modular strategy, and e\ is an entry node, it follows that the path (pl) u is also consistent 
with r, and has negative mean-payoff. In conclusion, we obtain that there exists a reachable negative 
non-decreasing cycle in A consistent with r, and this contradicts that r is a winning strategy. 

Hence, every path consistent with a does not contain a negative non-decreasing cycle. By Lemma 20 
it follows that a is a winning strategy in A for the objective LimlnfAvg > 0. □ 

Lemma 36 Let A be WRG. Player 1 has a modular winning strategy for the objective LimlnfAvg > 

iff there exists a memoryless modular winning strategy for player 1 for the objective. 
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Proof. The proof for the direction from right to left is trivial. The opposite direction is obtained 
as follows: by Lemma 26 it follows that if there is a modular winning strategy, then there is a cycle 
independent modular winning strategy; and by Lemma 35 it follows that if there is a cycle independent 
modular winning strategy, then there is a memoryless modular winning strategy. The desired result 
follows. □ 

We are now ready to prove the main result of this section. 

Theorem 8 The problem of deciding if player 1 has a modular winning strategy in a WRG A for the 
objective LimlnfAvg > is in NP. 

Proof. By Lemma 34 it is enough to guess a memoryless modular strategy (the memoryless modular 
strategy is the polynomial witness) and verify that it is indeed a winning strategy. The verification 
can be achieved in polynomial time using the polynomial time algorithms of Section 2 for WPSs with 
mean-payoff objectives. □ 

Strict inequalities and stack boundedness. Note that by Corollary 1 the results of Lemma 36 
and Theorem 8 also hold for objective LimSupAvg > 0. For modular strategies we only presented the 
result for mean-payoff objectives with non-strict inequalities. The results for strict inequalities follow 
from an adaptation of the proofs for non-strict inequalities (for which we prove memoryless modular 
strategies are sufficient). Moreover, the results also follow for mean-payoff objectives with the stack 
boundedness condition for the following reason: we observe that the manipulated operations never 
increase the stack height. Thus from our results it follow that if there is a modular winning strategy 
to ensure mean-payoff objective along with stack boundedness, then there is a memoryless modular 
strategy. Hence the NP upper bound follows for strict inequalities as well as for stack boundedness. 



4.3 NP-hardness of the modular winning strategy problem 

In this section we establish the NP-hardness of the modular winning strategy problem. Our hardness 
result will be for one-player WRGs (player- 1 WRGs), where every module will have single exit, and 
the weights are {— 1, 0, +1}. In other words, our hardness result show that even a very simple version 
of the problem (single exit one-player WRGs with constant weights) is NP-hard. 

Reduction. We present a reduction from the 3-SAT problem (satisfiability of a CNF for- 
mula where every clause has exactly three distinct literals). Consider a 3-SAT formula 
<p(xi,X2, ■ ■ ■ ,x n ) = /\™ =1 cl i , over n variables x\, X2, ■ ■ ■ , x n , and m clauses ch, cli, ■ ■ ■ , cl m . A 
literal is a variable x% or its negation -<Xi. We construct a player-1 WRG as follows: A v = 
(Aq, xi, ~>xi, X2, ~<X2 1 ■ ■ ■ , x n , ^x ni ch, c/2, . . • , cl m ) in the following way: there is an initial module 
A a , there is a module for every literal and for every clause. We now describe the modules. 

Module A . The module invokes in an infinite loop in sequence the modules cl\, cl 2 , ■ ■ ■ cl m , and all 
the transitions in this module have weight zero. 

Module for clause cli . There is an edge from the entry node of module cli to a box that invokes module 
y, for every literal y that appears in the clause ch. There is also an edge from the return node of y to 
the exit node of cli. All the weights in this module are zero. 

Module for literal yi . The entry node of yi has outdegree two (left edge and right edge) . The left edge 
is the FALSE edge, which leads to the exit node, and has a weight —1. The right edge is the TRUE 
edge, which leads to a box that invokes a call for module -it/j, and its weight is —1. The return of the 
box leads to the exit node and the edge weight is +2. The reduction is illustrated pictorially in Fig 5. 

Observation 2 Every path from the entry node of the module yi to its exit node, has a weight of at 
most 0, hence every path from the entry node of module ck to its exit node, has a weight of at most 0. 

Lemma 37 There exists a modular winning strategy for player 1 in A v for the objective LimlnfAvg > 
iff ip is satisfiable. 

Proof. We first observe that every modular strategy in A v is memoryless modular strategy. Observe 
that modular strategy for player 1 is a selection of a literal for every clause module, and selecting 
either the TRUE or FALSE edge for every literal module. We now present both directions of the proof. 
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Fig. 5. NP-hardness reduction 



— Modular winning strategy implies satisfiable. Let r be a modular winning strategy for player 1, 
such that the literal yj is chosen for clause ck. First, towards contradiction, let us assume that 
there exists i e {1, . . . , m} such that r selects the FALSE edge for module yj . Hence the weight 
of the path inside the module cli is negative. Thus, due to Observation 2, the mean-payoff value 
according to r is negative. Therefore r selects the TRUE edge for the module yj . Next, towards 
contradiction, let us assume that there exist i, j G {1, . . . , m} such that yj — -ij/J and r selects the 
TRUE edge for both modules. In this case, the play will never exit module yj , and will go forever 
through edges with negative weights. Therefore if r selects the TRUE edge for yj , then it does not 
select the TRUE edge for -iyj. Due to the above, the assignment that assigns a true value to the 
literal yj in clause cli is a valid (non-conflicting) assignment that satisfies ip. 

— Satisfiable implies modular winning strategy. Let 5 be a satisfying assignment (a non-conflicting 
assignment of truth values to variables) for the formula ip. We construct a modular winning 
strategy Tx as follows. In module cli, the modular strategy invokes the module yf , where yf is 
a literal for which x assigns a true value (since a; is a satisfying assignment such a literal must 
exist). In module ?/,, follow the TRUE edge if x assigns a true value to the literal yt, and follow the 
FALSE edge otherwise. It is straight forward to verify that the mean-payoff in a play according to 
Tx is zero. 

This completes the proof. □ 

Observe that in the hardness reduction we have used positive weight +2 for simplicity, which can be 
split into two edges of weight +1 each. Hence we have the following theorem. 

Theorem 9 The modular winning strategy problem is NP-hard for one-player WRGs (player- 1 
WRGs) with single exit for every module and objective LimlnfAvg > with edge weights in {—1, 0, +1}. 

Strict inequalities and stack boundedness. We first observe that the above reduction also holds 
for LimSupAvg > objective. Moreover, whenever the 3-SAT formula ip is satisfiable, then the witness 
memoryless modular strategy along with mean-payoff objective also ensures stack boundedness. Hence 
the hardness result follows for mean-payoff objectives with non-strict inequalities as well as for stack 
boundedness. The result for strict inequality is obtained as follows: we modify the above reduction by 
changing the weight of the edge back to the entry node of A from to 1. Then if the formula ip is 
satisfiable, then the average payoff for memoryless modular strategies is at least ^ , where |V| is the 
number of vertices, and if the formula ip is not satisfiable, then the mean-payoff under all memoryless 
modular strategies is at most 0. Hence the hardness follows also for mean-payoff objectives with strict 
inequalities. We have the following theorem summarizing the results for modular strategies. 

Theorem 10 The following assertions hold for WRGs with objectives & tx 0, for x€ {>,>}, ^ 6 
{LimSupAvg, LimlnfAvg}, as well as objectives <P along with stack boundedness. 



29 



1. If there is a modular winning strategy, then there is a memoryless modular winning strategy. 

2. The decision problem of whether there is a memoryless modular winning strategy is NP-complete. 

3. The decision problem is NP-hard for player-1 WRGs with single exit for every module and edge 
weights in {— 1,0, +1}. 



5 Conclusion 

In this work we study for the first time mean-payoff objectives in pushdown games and present a 
complete characterization of computational and strategy complexity. We show that pushdown systems 
(one-player pushdown games) with mean-payoff objectives under global strategies can be solved in 
polynomial time, whereas pushdown games with mean-payoff objectives under global strategics is 
undccidable. For modular strategies both pushdown systems and pushdown games with mean-payoff 
objectives are NP-complcte. We also show that global strategies for mean-payoff objectives in general 
require infinite memory even in pushdown systems; whereas memoryless strategies suffice for modular 
strategies for mean-payoff objectives. An interesting direction of future work would be to consider 
such games with multi-dimensional mean-payoff objectives. 
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